tobypde / frrn Goto Github PK

How do I do prediction in my own image?

Hello Tobias!

Thanks for a great paper!

I have a question about actual last layers dimensions in your work: in the paper you state that after last FRRU96 and streams concatenation RU48 goes — that means for me that we need such dimensionality reductor as 48 convolutions 1x1x(96+32) but in the FRRNABuilder::build code I see that after concatenation you add RU with self.base_channels + self.lanes to self.base_channels reductor:

network = self.add_ru(
    network, self.base_channels + self.lanes, self.base_channels)

In the add_ru() method if in_channels != out_channels the additional auto-reductor is added. Does it mean that lasagne in fact adds self.base_channels * self.multiplier + self.lanes to self.base_channels reductor before meaningful RU convolutions and that line in FRRNABuilder::build is just a false track?

Is this a version error incompatibility error? This is using the mypredict code for single image processing.

Using cuDNN version 5110 on context None
Mapped name None to device cuda0: GeForce GTX 1060 (0000:01:00.0)
2017-09-23 17:11:07,721 INFO Only predict function
Images 1
Traceback (most recent call last):
  File "predict.py", line 167, in <module>
    main()
  File "predict.py", line 142, in main
    predictions = pred_fn(image)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 917, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
RuntimeError: error getting worksize: CUDNN_STATUS_BAD_PARAM
Apply node that caused the error: GpuDnnConv{algo='small', inplace=True, num_groups=1}(GpuContiguous.0, GpuContiguous.0, GpuAllocEmpty{dtype='float32', context_name=None}.0, GpuDnnConvDesc{border_mode='half', subsample=(1, 1), dilation=(1, 1), conv_mode='conv', precision='float64', num_groups=1}.0, Constant{1.0}, Constant{0.0})
Toposort index: 757
Inputs types: [GpuArrayType<None>(float32, 4D), GpuArrayType<None>(float64, 4D), GpuArrayType<None>(float32, 4D), <theano.gof.type.CDataType object at 0x7f173b6f5b50>, Scalar(float32), Scalar(float32)]
Inputs shapes: [(1, 3, 256, 512), (48, 3, 5, 5), (1, 48, 256, 512), 'No shapes', (), ()]
Inputs strides: [(1572864, 524288, 2048, 4), (600, 200, 40, 8), (25165824, 524288, 2048, 4), 'No strides', (), ()]
Inputs values: ['not shown', 'not shown', 'not shown', <capsule object NULL at 0x7f1731d45a80>, 1.0, 0.0]
Outputs clients: [[GpuElemwise{Composite{(i0 * (Composite{(((i0 - i1) * i2 * i3) + i4)}(i1, i2, i3, i4, i5) + Abs(Composite{(((i0 - i1) * i2 * i3) + i4)}(i1, i2, i3, i4, i5))))}}[]<gpuarray>(GpuArrayConstant{[[[[ 0.5]]]]}, GpuDnnConv{algo='small', inplace=True, num_groups=1}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0)]]

Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "predict.py", line 167, in <module>
    main()
  File "predict.py", line 81, in main
    pred_fn = train.only_predict_function(network)
  File "/home/rjn/opencv3-p3-code/classification_and_boxing/newstuff/onhold/FRRN/FRRN/train.py", line 181, in only_predict_function
    network.output_layers, deterministic=True)[0]
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/helper.py", line 197, in get_output
    all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 333, in get_output_for
    conved = self.convolve(input, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 611, in convolve
    filter_flip=self.flip_filters)

HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

pip3.5 install theano==0.9.0

Collecting theano==0.9.0
Could not find a version that satisfies the requirement theano==0.9.0 (from versions: 0.3.0rc4, 0.3.0, 0.3.1rc1, 0.3.1rc2, 0.3.1, 0.4.0rc1, 0.4.0rc2, 0.4.0rc3, 0.4.0rc4, 0.4.0, 0.4.1rc1, 0.4.1rc2, 0.4.1, 0.5.0rc1, 0.5.0rc2, 0.5.0, 0.6.0rc1, 0.6.0rc2, 0.6.0rc3, 0.6.0rc5, 0.6.0, 0.7.0rc1, 0.7.0rc2, 0.7.0, 0.8.0rc1, 0.8.0, 0.8.1, 0.8.2)

No matching distribution found for theano==0.9.0

Getting this error, has anyone had this error and worked it out. I did open this issue on theano thread but no one seems to know what is going on.

>>> import theano
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10).  Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
 https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29

Using gpu device 0: GeForce GTX 1060 (CNMeM is enabled with initial size: 100 MB, cuDNN Mixed dnn version. The header is from one version, but we link with a different version (6021, 5005))

I tried tuning off cudnn with the line below..but FRRN seems to require cudnn and throws an error.
export THEANO_FLAGS='dnn.enabled=False'

Hi, I was wondering how effective the data augmentation is. Particularly, I was wondering if you observed significant performance difference on the validation and testing set, if without using the data augmentation. Thanks.

Hello!

I know your code is still somewhat in change, but would it be possible for you to list the version numbers of Theano and Lasagna you're using in the readme? I had to bounce around a bit. It appears that the code requires Lasagna 0.2, and Theano 0.9, but you're using their 'old' gpu backend?

Also, is there any way to run the code without cuDNN? If not, it's fine; I should really break down and buy a better GPU, but I thought I'd ask :)

What batch size did you use for training? I use a GeForce GTX 1080 Ti and can use a maximum batch size of 3. Isn't it better to use larger batch sizes for training on convolutional networks?

Is it possible to do multi gpu training and can I increase the batch size if I use multi gpu?

So many questions :) hope you can help me

When I trained the model with new semantic DB, I used empty folders("logs" and "models") for train.py.
For finish training process, I put Ctrl+C. Logs folder has the files for log. But there are no files in models folder.
Should I use the exising models file and folder for training(models/frrn_b.npz)?
Does the train.py makes new model file?
Do I have to get any initial stuff before train?
thanks

Hi there,
I'm really interested in using your network.
However I get this error:
2017-11-01 17:15:45,539 ERROR Cannot create cuDNN spatial log softmax. Install cuDNN and make sure that uses the GPU.
2017-11-01 17:15:45,540 INFO Chianti C++ library is not available.Will default to slower Python implementation.
I have updated theano and added " [lib]\n cnmem=1"to a .theanorc file in my home location.
I really need your help,so I beg your reply.

Hi there,

I wanted to test your network, but somehow I can't get ride of this error:

File "/media/---/Volume/---/FRRN/dltools/data.py", line 245, in
self._map_func = np.vectorize(lambda px: self._intensity_map[px])
TypeError: list indices must be integers, not NoneType

Do I miss something, or is there a problem with the code?
I use Anaconda environment with python 2.7 on Ubuntu 16.04 with Cuda 8.
The dependencies are installed:

python check_dependencies.py --cs_folder=cityscapes_lk
2017-08-05 10:23:13,954 INFO Found supported Python version 2.7.
2017-08-05 10:23:13,979 INFO Successfully imported numpy.
2017-08-05 10:23:13,998 INFO Successfully imported cv2.
2017-08-05 10:23:14,096 INFO Successfully imported sklearn.
2017-08-05 10:23:14,098 INFO Successfully imported sklearn.metrics.
2017-08-05 10:23:14,098 INFO Successfully imported scipy.
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
2017-08-05 10:23:14,223 WARNING The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
Using gpu device 1: GeForce GTX 1080 (CNMeM is enabled with initial size: 95.0% of memory, cuDNN 5110)
2017-08-05 10:23:15,236 INFO Successfully imported theano.
2017-08-05 10:23:15,239 INFO Successfully imported lasagne.
2017-08-05 10:23:15,239 INFO Theano float is float32.
2017-08-05 10:23:15,240 INFO cuDNN spatial softmax found.
2017-08-05 10:23:15,241 INFO Chianti C++ library is not available. Will default to slower Python implementation.
2017-08-05 10:23:15,249 INFO Found CityScapes training set.
2017-08-05 10:23:15,250 INFO Found CityScapes validation set.

Full Traceback until ctrl-c:

python predict.py --architecture frrn_a --model_file models/frrn_a.npz --cs_folder cityscapes_lk
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
Using gpu device 1: GeForce GTX 1080 (CNMeM is enabled with initial size: 95.0% of memory, cuDNN 5110)
2017-08-05 10:23:23,614 INFO Compile validation function
Process Process-2:
Traceback (most recent call last):
File "/home/---/anaconda3/envs/frnn/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/---/anaconda3/envs/frnn/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/media/---/Volume/---/FRRN/dltools/data.py", line 70, in init
self.start_loading()
File "/media/---/Volume/---/FRRN/dltools/data.py", line 78, in start_loading
images, targets = self.load_batch()
File "/media/---/Volume/---/FRRN/dltools/data.py", line 87, in load_batch
data = [self._data_loader.get_pair() for _ in range(self._batch_size)]
File "/media/---/Volume/---/FRRN/dltools/data.py", line 47, in get_pair
target = self._target_loader.load(target_file)
File "/media/---/Volume/---/FRRN/dltools/data.py", line 257, in load
img = self._map_func(img)
File "/home/---/anaconda3/envs/frnn/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2734, in call
return self._vectorize_call(func=func, args=vargs)
File "/home/---/anaconda3/envs/frnn/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2804, in _vectorize_call
ufunc, otypes = self._get_ufunc_and_otypes(func=func, args=args)
File "/home/---/anaconda3/envs/frnn/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2764, in _get_ufunc_and_otypes
outputs = func(*inputs)
File "/media/---/Volume/---/FRRN/dltools/data.py", line 245, in
self._map_func = np.vectorize(lambda px: self._intensity_map[px])
TypeError: list indices must be integers, not NoneType

Thanks in advance!

Trying to evaluation with my own image....not sure how to set the inputs correctly. Can you offer some correction.

        im = cv2.imread('./eagle.jpg')
        im     = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)         
        imgray = cv2.cvtColor(im, cv2.COLOR_RGB2GRAY)
        imgray =  cv2.normalize(imgray, dst=imgray, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)[0:-1]
        imgray = imgray[np.newaxis,...]
               
        batch1 = [im]
        batch2 = [imgray] 
        predictions, loss = val_fn(batch1, batch2)

ERROR:

Traceback (most recent call last):
  File "cam_predict.py", line 90, in <module>
    main()
  File "cam_predict.py", line 74, in main
    predictions, loss = val_fn([batch1], [batch2])
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 917, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
ValueError: images and kernel must have the same stack size


Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "cam_predict.py", line 90, in <module>
    main()
  File "cam_predict.py", line 62, in main
    val_fn = train.compile_validation_function(network, BATCH_SIZE)
  File "/home/rjn/opencv3-p3-code/classification_and_boxing/newstuff/FRRN/train.py", line 153, in compile_validation_function
    network.output_layers, deterministic=True)[0]
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/helper.py", line 197, in get_output
    all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 333, in get_output_for
    conved = self.convolve(input, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 611, in convolve
    filter_flip=self.flip_filters)

Hi @TobyPDE

I can inference this network and result is good, but there are some issue in train.py

This is my log.

Using gpu device 0: GeForce GTX 970 (CNMeM is disabled, cuDNN 5005)
Enter path to CityScapes folder [DataSet/]: 
Enter model filename [models/frrn_b.npz]: 
Enter log filename [logs/frrn_b.log]: 
Define network... [0.32s]
Load model... [0.26s]
Define loss... [1.54s]
Compile update functions... [158.36s]
Compile validation function... [22.07s]
Optimize...Evaluate block 5/5
 [0.70s]
Traceback (most recent call last):
  File "train.py", line 197, in <module>
    optimizer.optimize()
  File "/home/FRRN/dltools/optimizer.py", line 55, in optimize
    losses = self.train_fn(*train_fn_args)
  File "train.py", line 140, in compute_update
    loss, grads = dltools.hybrid_training.compute_grads(grad_fns, param_blocks, *args)
  File "/home/unicornx/FRRN/dltools/hybrid_training.py", line 112, in compute_grads
    result = grad_fns[i](*args, *prev)
  File "/home/Theano/theano/compile/function_module.py", line 788, in __call__
    allow_downcast=s.allow_downcast)
  File "/home/Theano/theano/tensor/type.py", line 178, in filter
    data.shape))
TypeError: Bad input argument to theano function with name "/home/FRRN/dltools/hybrid_training.py:84" at index 1 (0-based).  
Backtrace when that variable is created:

  File "train.py", line 74, in <module>
    target_var = T.itensor3()
Wrong number of dimensions: expected 3, got 4 with shape (3, 3, 512, 1024).

Any idea?
Thanks!!

First, thanks for the new update.
I do not have access to Cityscape, how can I use the pre trained models on other images?
Not sure how to transform pics and how to input them to the net properly.

thanks

Tets

You might, if you have time, consider adding in associative memory to the structure of your neural networks on the presumption that it will give them extra "powers".
Associative memory was a serious topic in the 1960's (see archive.org) through the 1970's and up to the time of Hopfield.
Unfortunately all those researchers missed a very simple simple algorithm for doing associative memory:
https://devtalk.nvidia.com/default/topic/1023786/cuda-programming-and-performance/walsh-hadamard-transform-based-ai/post/5208577/#5208577
The idea is to provide the deep network with "soft" memory that it can actually learn to use.

Is there any number of iteration of training?
or Just put Ctrl+C to stop training?

@TobyPDE
The batch_size in test_classification_loss is same to the batch_size in train_loss, which is default by 3 different to the value 1 in validation_provider.

    # Validation classification loss (supervised)
    test_classification_loss = dltools.utility.bootstrapped_categorical_cross_entropy4d_loss(
        test_predictions,
        target_var,
        batch_size = config["batch_size"],
        multiplier = 64)

    validation_provider = dltools.data.CityscapesHDDDataProvider(
        config["cityscapes_folder"],
        file_folder="val",
        batch_size=1,
        augmentor=None,
        sampling_factor=config["sample_factor"],
        random=False
    )

Hi, @TobyPDE ,

Could you list the version for each prerequisite package you mentioned in https://github.com/TobyPDE/FRRN#prerequisite?

Thanks!

Hi,

I am trying to run "python predict.py" and when I have to enter the model name I enter ""/home/user/Documents/FRRN-master/models/frrn_p.npz" and I get always the error "TypeError: super() takes at least 1 argument (0 given)"

How should I enter this path? I am using ubuntu 14.04 distribution

Thanks in advance and best regards
Jugaris

Hey, thanks for sharing the code. This is a fantastic idea to do semantic segmentation.

May I ask: is there a way to utilize your code on other data, like my own data ? Is there an instruction for this purpose ?

Best
He Zhao

could you provide the init.model to train?

Thanks for your greate work, have you ever test your algorithm on PASCAL VOC or COCO datasets?

Hi, @TobyPDE ,

I installed the prerequisite packaged as listed in readme.md, then I cloned FRRN and then issue command:
python predict.py, I got the following error :

(cv_python3) root@milton-OptiPlex-9010:/data/code2/FRRN# python predict.py /root/.virtualenvs/cv_python3/lib/python3.4/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
  "downsample module has been moved to the theano.tensor.signal.pool module.")
Traceback (most recent call last):
  File "predict.py", line 3, in <module>
    import dltools
  File "/data/code2/FRRN/dltools/__init__.py", line 12, in <module>
    from . import hybrid_training
  File "/data/code2/FRRN/dltools/hybrid_training.py", line 112
    result = grad_fns[i](*args, *prev)
                                ^
SyntaxError: invalid syntax
(cv_python3) root@milton-OptiPlex-9010:/data/code2/FRRN#

Any suggestion to fix it? Thx~

I have problem when running the train.py, it shows me each time an error 'cannot import name 'downsample' and when i change the version of theano and Lasagne i have a problem with gpuarray.

Can you please cite the exact version of Theano and Lasagne.

Thank you

I want to adapt the the resolution to 1280x800. Therefore I changed the IMAGE_ROWS and IMAGE_COLS in the train.py script and resized the train and val images. The training works but for prediction I get an "Segmentation fault (core dumped)" error. The images for prediction are also adapted to 1280x800. I tried also different sample sizes.

any suggestions?

Thanks

Hi there,
I'm really interested in using your network and performance does matter, that's why I installed chianti.
However I get this error:

python predict.py --architecture frrn_a --model_file models/frrn_a.npz --cs_folder ../Datasets/cityscapes_lk
Using cuDNN version 6021 on context None
Mapped name None to device cuda0: GeForce GTX 960 (0000:01:00.0)
2017-08-11 11:30:03,314 INFO Compile validation function
Traceback (most recent call last):
File "predict.py", line 84, in
main()
File "predict.py", line 63, in main
BATCH_SIZE)
File "/---/FRRN/train.py", line 264, in get_validation_provider
pychianti.Iterator.Sequential(validation_images),
TypeError: No registered converter was able to produce a C++ rvalue of type std::__cxx11::basic_string<char, std::char_traits, std::allocator > from this Python object of type str

I'm using python 2.7 in Anaconda. I have all dependencies. Boost version is 1.64. I tried different version of boost / boost-python and still got the same error. Any clue?

It seems your old github didn't use chinati library? Why you write a separate C++ library?

Hello! Recently I encountered a problem when I ran your Networks.
When I ran the predict.py, it came out as follow:
Traceback (most recent call last):
File "/home/dingbaojin/anaconda2/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
self.run()
File "/home/dingbaojin/anaconda2/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/dingbaojin/deep_learning/Full_Resolution_Residual_Networks_test/FRRN-master/dltools/data.py", line 70, in init
self.start_loading()
File "/home/dingbaojin/deep_learning/Full_Resolution_Residual_Networks_test/FRRN-master/dltools/data.py", line 78, in start_loading
images, targets = self.load_batch()
File "/home/dingbaojin/deep_learning/Full_Resolution_Residual_Networks_test/FRRN-master/dltools/data.py", line 91, in load_batch
targets = [self._convert_target(d[1]) for d in data]
File "/home/dingbaojin/deep_learning/Full_Resolution_Residual_Networks_test/FRRN-master/dltools/data.py", line 125, in _convert_target
selects = class_matrix[flat_targets]
IndexError: index 70 is out of bounds for axis 0 with size 20
I hope your help, Thanks a lot!

Hi @TobyPDE ,

I have some training data was labeled in color, I would like to transform color image to labelIds image for training.
label.py have the mapping table, so I just using 'id' to replace [r,g,b] and save one channel to image.
Is this make sense?

Any idea?
Thanks.

Hi !@TobyPDE
I installed all the prerequisites as mentioned in README file. But i still get an error when i try to run predict_frrn_a.py
Please help..

(frrn) mrl@mrl:~/FRRN$ python predict_frrn_a.py
Enter path to CityScapes folder [/home/mrl/FRRN/leftImg8bit_trainvaltestt/leftImg8bit]: /home/mrl/FRRN/cityscapes
Enter model filename [/home/mrl/FRRN/models/frrn_a.npz]: /home/mrl/FRRN/models/frrn_a.npz
Define network... [0.34s]
Load model... [0.37s]
Compile validation function... [0.27s]
Traceback (most recent call last): File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/type.py", line 267, in dtype_specs }[self.dtype] KeyError: 'object'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 408, in constant_or_value TensorType(dtype=x_.dtype, broadcastable=bcastable), File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/type.py", line 50, in init self.dtype_specs() # error checking is done there File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/type.py", line 270, in dtype_specs % (self.class.name, self.dtype)) TypeError: Unsupported dtype for TensorType: object
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 202, in as_tensor_variable return constant(x, name=name, ndim=ndim) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 422, in constant dtype=dtype) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 417, in constant_or_value raise TypeError("Could not convert %s to TensorType" % x, type(x)) TypeError: not all arguments converted during string formatting
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "predict_frrn_a.py", line 66, in test_predictions = lasagne.layers.get_output(network.output_layers, deterministic=True)[0] File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/lasagne/layers/helper.py", line 190, in get_output all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs) File "/home/mrl/FRRN/dltools/layers.py", line 36, in get_output_for return T.nnet.abstract_conv.bilinear_upsampling(input, self.factor) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/nnet/abstract_conv.py", line 569, in bilinear_upsampling row * ratio, col * ratio)) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/var.py", line 327, in reshape return theano.tensor.basic.reshape(self, shape, ndim=ndim)
File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 4526, in reshape newshape = as_tensor_variable(newshape) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 208, in as_tensor_variable raise AsTensorError("Cannot convert %s to TensorType" % str_x, type(x)) theano.tensor.var.AsTensorError: ('Cannot convert (None, None, Elemwise{mul,no_inplace}.0, Elemwise{mul,no_inplace}.0) to TensorType', <class 'tuple'>)

Hi @TobyPDE

I have use train.py to train a new model, but if I would like to fine tune the pre trained model, how should I do?

Thanks!!

I'd like to train on 400x600 images, but I am encountering a problem.

ValueError: Mismatch: not all input shapes are the same

is raised in https://github.com/TobyPDE/FRRN/blob/master/dltools/architectures.py#L315 because autobahn.input_shapes = [(None, 32, 600, 400), (None, 32, 592, 400)]
Are there any constraints on the input dimensions?

Thank you so much for your open source code.

I run into problems when install theano and lasagne on python3.5.1.

First, I install python 3.5.1 from source code.
Then, I use pip3.5 install theano and lasagne.

Here is the error message
Problem occurred during compilation with the command line below:
/usr/bin/g++ -shared -g -march=core-avx2 -mcx16 -msahf -mmovbe -maes -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mrtm -mhle -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=51200 -mtune=generic -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/usr/local/lib/python3.5/site-packages/numpy/core/include -I/usr/local/include/python3.5m -I/usr/local/lib/python3.5/site-packages/theano/gof -fvisibility=hidden -o /root/.theano/compiledir_Linux-4.2--generic-x86_64-with-debian-jessie-sid-x86_64-3.5.1-64/lazylinker_ext/lazylinker_ext.so /root/.theano/compiledir_Linux-4.2--generic-x86_64-with-debian-jessie-sid-x86_64-3.5.1-64/lazylinker_ext/mod.cpp -L/usr/local/lib -lpython3.5m
/usr/bin/ld: /usr/local/lib/libpython3.5m.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libpython3.5m.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

Operating System :

Ubuntu 14

2 warning and 1 error
Using gpu device 0: GeForce GTX 1080 Ti (CNMeM is disabled, cuDNN 6021)
/usr/local/lib/python2.7/dist-packages/theano/sandbox/cuda/init.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
warnings.warn(warn)

/usr/local/lib/python2.7/dist-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
"downsample module has been moved to the theano.tensor.signal.pool module.")

2017-09-08 19:34:11,781 ERROR Cannot create cuDNN spatial log softmax. Install cuDNN and make sure that theano uses the GPU.

need your help

I tried to run FRRN using other pictures for prediction. But I had to change the names of pictures and the names had to be just like the pictures of Cityscapes. Can you offer a version that FRRN can predict all pictures without change the names of pictures?
I tried to change the code to realize this idea but the results were so bad that I doubted that there was something wrong with my code. Once I removed the "batch[1]" from the "val_fn" in prediction.py and, of course, correspondingly changed the code in train.py, the results changed and the results got bad.

Hey, I tested your framework with the cityscape dataset and the trained models (predict.py) but the output is very bad (losses between 100 - 500). I checked the output images(image, gt_img and pred_img) and saw that the RGB is just black. The gt is the same like from the dataset and the pred is a very bad segmented image.

The paths to the cityscape seems to be right and the dependency check runs with no error.

Thank you for help.

Hi, @TobyPDE

After switching to install python3.5, I pass the command "python train.py". However, after inputing "path to ityScapes folder", "model filename", "log filename", it runs into the following troubles:

(root) root@milton-All-Series:/data/code/FRRN# python train.py 
Enter path to CityScapes folder [/data/CityScrapes_dataset/FRRN_data/leftImg8bit_trainvaltest/train]: /data/CityScrapes_dataset/FRRN_data/leftImg8bit_trainvaltest/train
Enter model filename [models/frrn_b.npz]: 
Enter log filename [logs/frrn_b.log]: 
Define network... [0.46s]
Load model... [0.90s]
Define loss... [0.88s]
Traceback (most recent call last):
  File "train.py", line 78, in <module>
    all_predictions, split_outputs = dltools.hybrid_training.get_split_outputs(network, deterministic=False)
  File "/data/code/FRRN/dltools/hybrid_training.py", line 18, in get_split_outputs
    all_outputs = lasagne.layers.get_output(network.output_layers + sum(network.splits, []), **kwargs)
  File "/root/anaconda3/lib/python3.5/site-packages/lasagne/layers/helper.py", line 191, in get_output
    all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
  File "/root/anaconda3/lib/python3.5/site-packages/lasagne/layers/special.py", line 52, in get_output_for
    return self.nonlinearity(input)
  File "/data/code/FRRN/dltools/architectures.py", line 88, in log_softmax_4d
    x = theano.sandbox.cuda.basic_ops.gpu_contiguous(x)
  File "/root/anaconda3/lib/python3.5/site-packages/theano/gof/op.py", line 604, in __call__
    node = self.make_node(*inputs, **kwargs)
  File "/root/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/basic_ops.py", line 3910, in make_node
    input = as_cuda_ndarray_variable(input)
  File "/root/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/basic_ops.py", line 46, in as_cuda_ndarray_variable
    return gpu_from_host(tensor_x)
  File "/root/anaconda3/lib/python3.5/site-packages/theano/gof/op.py", line 604, in __call__
    node = self.make_node(*inputs, **kwargs)
  File "/root/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/basic_ops.py", line 132, in make_node
    dtype=x.dtype)()])
  File "/root/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/type.py", line 95, in __init__
    (self.__class__.__name__, dtype, name))
TypeError: CudaNdarrayType only supports dtype float32 for now. Tried using dtype float64 for variable None

Any suggestion to fix it?

Here are my packages installed:
python 3.5.2
opencv 3.1.0
Theano 0.9.0.dev4
Lasagne 0.2.dev1
numpy 1.11.2
scikit-learn 0.17.1

Thanks~

Define network...
[16.67s]
Load model... [0.33s]
Define loss... [1.14s]
Compile update functions... [82.92s]
Compile validation function... [14.34s]
Optimize... [3.68s]
Traceback (most recent call last):
File "/raid/code/FRRN/FRRN-master/train_frrn_a.py", line 213, in
optimizer.optimize()
File "/raid/code/FRRN/FRRN-master/dltools/optimizer.py", line 58, in optimize
losses = self.train_fn(data.imgs, data.targets, update_counter)
File "/raid/code/FRRN/FRRN-master/train_frrn_a.py", line 141, in compute_update
loss, grads = dltools.hybrid_training.compute_grads(grad_fns, param_blocks, imgs, targets)
File "/raid/code/FRRN/FRRN-master/dltools/hybrid_training.py", line 150, in compute_grads
result = grad_fnsi
File "/usr/local/lib/python3.4/dist-packages/theano/compile/function_module.py", line 867, in call
self.inv_finder[c]))
TypeError: Missing required input: <TensorType(float32, 4D)>

Process finished with exit code 1