Could not load ResNet model about fast-neural-style HOT 15 CLOSED

fast-neural-style expects Torch checkpoints instead of Caffe checkpoints; if you want to use ResNets then you should use these model checkpoints:

https://github.com/facebook/fb.resnet.torch/tree/master/pretrained

from fast-neural-style.

@jcjohnson Do you know of any projects I could use to convert .caffemodels to t7 .models?

from fast-neural-style.

@ProGamerGov This script of mine uses loadcaffe and can convert some Caffe models to Torch for use in fast-neural-style, but it won't work with models like ResNet or GoogLeNet:

https://github.com/jcjohnson/cnn-benchmarks/blob/master/convert_model.lua

from fast-neural-style.

@jcjohnson So the traditional Caffe models that worked with the Classic (Original) Neural-Style, should work for conversion to Torch models?

I setup cnn-benchmarks, but I seem to receive the same error with any model I try to convert: jcjohnson/cnn-benchmarks#11

from fast-neural-style.

Yes, any model that worked with original neural-style should be convertable to Torch format using the conversion script.

from fast-neural-style.

I think this is resolved?

from fast-neural-style.

@jcjohnson Sorry for asking a question that is mostly unrelated to this issue, but what is the difference between .t7 and .t7b models? I'd like to convert your pre-trained model from: https://github.com/jcjohnson/densecap from it's .t7 format to a caffemodel format for use in Neural-Style, and the various model conversion guides I have found, use a .t7b model instead of a .t7 model.

from fast-neural-style.

@ProGamerGov I don't think there is much of a difference; I'd guess that maybe .t7b is to emphasize that the model was saved in binary mode rather than ascii mode, but in practice almost all Torch checkpoints are saved in binary mode anyway since it is much more efficient.

BTW if you want to use the densecap model in neural-style then converting .t7 to caffemodel back to Torch seems overly complicated; it might be easier just to tweak neural-style so it can work directly with a .t7 file; this is for example how slow_neural_style.lua works.

from fast-neural-style.

@jcjohnson Thanks for the clarification on the Torch7 model file extensions.

I think you may be right that modifying Neural-Style is my best bet. I tried to convert the model using fb-caffe-exts, but that resulted in the error: unknown Torch class <DenseCapModel>, which I posted in this issue on the project. I'm not sure how difficult solving this issue is, and whether or not it's worth it to try and resolve the issue. Would this error indicate that there would be any issues with using the model in Neural-Style?

How difficult would it be for someone to implement the .t7 model support from slow_neural_style.lua into neural_style.lua? How similar are the two?

from fast-neural-style.

@ProGamerGov

Adding .t7 model support to neural-style should be pretty straightforward; the main difference is that .t7 models don't have string layer names, so you'd need to index the style and content layers by index like in slow_neural_style.lua.

from fast-neural-style.

There was a pull request by @szagoruyko last March which allows loading a .t7 model into neural-style jcjohnson/neural-style#169 . I used it back then to try out ResNet.

from fast-neural-style.

As to the unknown Torch class issue, it happens all the time nowadays when trying to use models across projects (projects have implemented their own classes in addition to the standard ones). The solution is to copy the implementation of the missing class into the project where it is needed and require it (and hope that there are no conflicts).

I guess importing the missing class implementation would not help when converting into Caffe format, as Caffe would not understand the additional class anyway, but it would probably work in neural-style which is running in Torch.

from fast-neural-style.

@htoyryla Using @szagoruyko's Neural-Style modification here: jcjohnson/neural-style#169, results in the same issue as FB's Torch2Caffe converter.

/home/ubuntu/torch/install/bin/luajit: /home/ubuntu/torch/install/share/lua/5.1/
torch/File.lua:343: unknown Torch class <DenseCapModel>
stack traceback:
        [C]: in function 'error'
        /home/ubuntu/torch/install/share/lua/5.1/torch/File.lua:343: in function 'readObject'
        /home/ubuntu/torch/install/share/lua/5.1/torch/File.lua:369: in function  'readObject'
        /home/ubuntu/torch/install/share/lua/5.1/torch/File.lua:409: in function  'load'
        neural_style_t7.lua:77: in function 'main'
        neural_style_t7.lua:526: in main chunk
        [C]: in function 'dofile'
        ...untu/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
        [C]: at 0x00405d50

from fast-neural-style.

Did you read my second comment? The .t7 file refers to Torch objects, most of them may be standard but some of them can have been defined in the project which created the t7 file, like here DenseCapModel. It is not a standard Torch class, it has been implemented by @jcjohnson in https://github.com/jcjohnson/densecap/blob/master/densecap/DenseCapModel.lua
The only way to make this work is to import the missing class (and possibly other required parts from DenseCap project) into neural-style. See also my earlier comment.

from fast-neural-style.

@htoyryla Sorry, my bad. I guess posting the same error twice was redundant. I understand that I would need to implement the DenseCap module into the Neural-Style code in some form to make it work.

from fast-neural-style.