imlab-uiip / keras-segnet Goto Github PK

Hi

Thank you for sharing the code. I have been trying to apply your code a data set, which has image size of 128*128. However, running the code gives the following error message,

ValueError: Cannot feed value of shape (32, 1, 128, 128) for Tensor 'activation_26_target:0', which has shape '(?, ?, ?)' Would you like to share any thought on the possible reason? Is that because the current implementation can only accept images with 256. If the image is of other sizes, we have to change the architecture. Thanks.

The following is the whole traceback

File "/development/tfw/lib/python3.4/site-packages/Keras-1.0.3-py3.4.egg/keras/models.py", line 409, in fit
    sample_weight=sample_weight)
  File "/development/tfw/lib/python3.4/site-packages/Keras-1.0.3-py3.4.egg/keras/engine/training.py", line 1052, in fit
    callback_metrics=callback_metrics)
  File "/development/tfw/lib/python3.4/site-packages/Keras-1.0.3-py3.4.egg/keras/engine/training.py", line 790, in _fit_loop
    outs = f(ins_batch)
  File "/development/tfw/lib/python3.4/site-packages/Keras-1.0.3-py3.4.egg/keras/backend/tensorflow_backend.py", line 659, in __call__
    updated = session.run(self.outputs + self.updates, feed_dict=feed_dict)
  File "/development/tfw/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 340, in run
    run_metadata_ptr)
  File "/development/tfw/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 553, in _run
    % (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (32, 1, 128, 128) for Tensor 'activation_26_target:0', which has shape '(?, ?, ?)'

I am trying to run this in windows python 3 with tensor`flow backend(somehow my theano backend fail to utilize gpu). i converted print syntax in the ipython notebook and tried to run the script, it went well, but when i reached to "Loading and compiling model built with 'build_model.py'", received the following error.
May I know what needs to be changed, based on this error. Also, what would be your suggestion if I would like to run the ipython using tensorflow backend with python 3

`thank you in advance for looking into this issue.

Hi,
Your code is of great value for people who are new to DL in Keras and Segnet, and I really appreciate your work. I found you use function prep_data(mode) to make label for the training and testing dataset, but there's not any example illustrating what's the .CSV file should look like, could you just kindly give an example of your training dataset and the CSV file corresponding to it?

Thank you,
Tony

Hi,
I have generated the training data with corresponding ground truths and the required ".csv" file. How can I train it from scratch using my data ? Can you please elaborate the steps ?
Thanks !

Seem in the last layers you reshape all to [2,65536] (assuming this is row x col) and 256*256 is 65536, so it's just image with 2 'channels' (assuming background/foreground) reshaped to flat vector. So in general case it will be [n_classes, h*w]. And than softmax is putted on top.
https://github.com/imlab-uiip/keras-segnet/blob/master/SegNet.py#L19
https://github.com/imlab-uiip/keras-segnet/blob/master/model_5l.json#L1266

Also here you use categorical_crossentropy loss.
https://github.com/imlab-uiip/keras-segnet/blob/master/SegNet.py#L108

My confusion is how softmax is applied and how loss is computed, as I understand it should be like:
compute softmax for each [2,i] (and not to [i,65536]) subvector separately of matrix and loss should be computed for each pixel, so as I understand Keras just do it all in some vectorized form?
So what is the rules to pass matrix/tensor to softmax+categorical_crossentropy? what shape shoul it be? row represent dimensions and cols represent pixels?
Documentation is not rich about this topics:
https://keras.io/activations/#softmax
https://keras.io/losses/#categorical_crossentropy

BTW what is profit of store model as .json ? I find this solution less readable.

I ran the code as it is : It generates me the plotted output : but The thing is The output comes as an empty one . on all 3 instances of Test Image .
I have attached my screenshot of the result.png file . I haven't made even a single modification in the code . What might be wrong ?

Hi,
as far as i understand, this implementation does not use the index-based upsampling, but uses the Upsample2D Keras layer. Do you have any ideas on how to implement the index passing and how much of the impact this should have?

Thanks!
Kind regards,
Tanja

Ran into this issue when the following command is run: python SegNet.py

/usr/lib/python2.7/dist-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
Using Theano backend.
Using gpu device 0: GeForce GTX 1070 (CNMeM is disabled, cuDNN 5105)
/usr/local/lib/python2.7/dist-packages/theano/sandbox/cuda/__init__.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
  warnings.warn(warn)
Traceback (most recent call last):
  File "SegNet.py", line 105, in <module>
    autoencoder = models.model_from_json(model_file.read())
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 210, in model_from_json
    return layer_from_config(config, custom_objects=custom_objects)
  File "/usr/local/lib/python2.7/dist-packages/keras/utils/layer_utils.py", line 40, in layer_from_config
    return layer_class.from_config(config['config'])
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 1080, in from_config
    model.add(layer)
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 327, in add
    output_tensor = layer(self.outputs[0])
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 569, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 632, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 168, in create_node
    output_shapes = to_list(outbound_layer.get_output_shape_for(input_shapes[0]))
  File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 336, in get_output_shape_for
    self.target_shape)
  File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 330, in _fix_unknown_dimension
    raise ValueError(msg)
ValueError: total size of new array must be unchanged

Got the following error when I run the following command: python SegNet.py

Using Theano backend.
Using gpu device 0: GeForce GTX 1070 (CNMeM is disabled, cuDNN 5105)
/usr/local/lib/python2.7/dist-packages/theano/sandbox/cuda/__init__.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
  warnings.warn(warn)
Compiled: OK
Traceback (most recent call last):
  File "SegNet.py", line 118, in <module>
    autoencoder.load_weights('model_5l_weight_ep50.hdf5')
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 2695, in load_weights
    f = h5py.File(filepath, mode='r')
  File "/usr/lib/python2.7/dist-packages/h5py/_hl/files.py", line 272, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "/usr/lib/python2.7/dist-packages/h5py/_hl/files.py", line 92, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/build/h5py-nQFNYZ/h5py-2.6.0/h5py/_objects.c:2577)
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/build/h5py-nQFNYZ/h5py-2.6.0/h5py/_objects.c:2536)
  File "h5py/h5f.pyx", line 76, in h5py.h5f.open (/build/h5py-nQFNYZ/h5py-2.6.0/h5py/h5f.c:1811)
IOError: Unable to open file (File signature not found)

Hi, I get the following error when loading the json model. I have tried in both Keras vs 1 and 2 and I run with Theano as backend.

File "", line 1, in
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/models.py", line 345, in model_from_json
return layer_module.deserialize(config, custom_objects=custom_objects)
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/layers/init.py", line 54, in deserialize
printable_module_name='layer')
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/utils/generic_utils.py", line 139, in deserialize_keras_object
list(custom_objects.items())))
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/models.py", line 1214, in from_config
model.add(layer)
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/models.py", line 475, in add
output_tensor = layer(self.outputs[0])
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/engine/topology.py", line 621, in call
output_shape = self.compute_output_shape(input_shape)
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/layers/core.py", line 376, in compute_output_shape
input_shape[1:], self.target_shape)
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/layers/core.py", line 370, in _fix_unknown_dimension
raise ValueError(msg)
ValueError: total size of new array must be unchanged

Hi, with respect to this segnet model, we can see that in each building block of encoder part, there are always two convolutional layers following each other before a maxpooling layer. For instance,

Convolution2D(64, kernel, kernel, border_mode='same', input_shape=(1, img_h, img_w)),
    BatchNormalization(),
    Activation('relu'),
    Convolution2D(64, kernel, kernel, border_mode='same'),
    BatchNormalization(),
    Activation('relu'),
    MaxPooling2D(),

I am not very clear about the real logic of having two convolutional layers following each other. Why not just having a combined convolutional layer. What's the strength of having this kind of design shown in Segnet architecture.

Dear all,

I wish to apply Segnet Model for image segmentation task. These are medical images. I want to initialize the weights for Segnet model using Camvid dataset. What changes can I do to the current model to initialize the weights? I saw some thread https://github.com/legokichi/keras-segnet/blob/master/model_segnet.py, where they initialize weights using VGG16 pre-trained on Camvid dataset. But I couldn't really run that script (some error).

Any help would be appreciated.

Thank you

Hi @eclique,
Could you tell us what versions of python packages you are using?
Many thanks
Ian

Hi, thanks for sharing the code.
I have two questions:

1 tensorflow backend. I use tensorflow back to train the model, But the predicted output is poor.
2 how to use your index-based-upsampling code? If I use tensorflow backend, how do I modify it?

Thank you.

I tried pulling the .hdf5 file using git lfs, but I am getting this error.

batch response: This repository is over its data quota. Purchase more data packs to restore access.

Is there any other way to download the file?

I am trying to convert the build.py to python 3 and keras version 2.1.4. I came a long way but I have some trouble with (hopefully) the last step. Below I posted the code I have now but what I need to do yet is the final reshaping. I receive the error ValueError: total size of new array must be unchanged which according to this answer I got this error because the shape of my output is not corresponding to what I want to reshape it to. However my height and width are both factors of 32 so I do not understand the problem. The final layer of the network (a batch normalization) returns a shape of (Bat (None, 0, 256, 2). According to the code I found I have to reshape this using

autoencoder.add(Reshape((n_labels, img_h*img_w))

which would be (2, (256 * 256)). This sounds incorrect the 256 * 256 part is very large, but I don't fully understand the reshape function.

I saw similair questions in other threads but there the problem seems to be that tensorflow was used instead of theano. I am using Theano as backend.

Can somebody shed some light on what this reshaping does and maybe suggest to what dimensions I need to reshape?

This is the entire model I'm using:

from keras import models
from keras.layers.core import Activation, Reshape, Permute
from keras.layers.convolutional import Conv2D, MaxPooling2D, UpSampling2D
from keras.layers.normalization import BatchNormalization
import json

img_w = 256
img_h = 256
n_labels = 2

kernel = 3

encoding_layers = [
    Conv2D(64, (kernel,kernel), padding='same', input_shape=(1, img_h, img_w)),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(64, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    MaxPooling2D(),

    Conv2D(128, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(128, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    MaxPooling2D(),

    Conv2D(256, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(256, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(256, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    MaxPooling2D(),

    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    MaxPooling2D(),

    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    MaxPooling2D(),
]

autoencoder = models.Sequential()
autoencoder.encoding_layers = encoding_layers

for l in autoencoder.encoding_layers:
    autoencoder.add(l)

decoding_layers = [
    UpSampling2D(),
    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),

    UpSampling2D(),
    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(512, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(256, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),

    UpSampling2D(),
    Conv2D(256, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(256, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(128, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),

    UpSampling2D(),
    Conv2D(128, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(64, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),

    UpSampling2D(),
    Conv2D(64, (kernel,kernel), padding='same'),
    BatchNormalization(),
    Activation('relu'),
    Conv2D(n_labels, (1, 1), padding='valid'),
    BatchNormalization(),
]
autoencoder.decoding_layers = decoding_layers
for l in autoencoder.decoding_layers:
    autoencoder.add(l)

autoencoder.add(Reshape((n_labels, img_h * img_w)))
autoencoder.add(Permute((2, 1)))
autoencoder.add(Activation('softmax'))

with open('model_5l.json', 'w') as outfile:
    outfile.write(json.dumps(json.loads(autoencoder.to_json()), indent=2))

hi,i am running build_model.py,it raises ValueError: Negative dimension size caused by subtracting 2 from 1 for 'max_pooling2d_2/MaxPool' (op: 'MaxPool') with input shapes: [?,1,1,256].i have tried to set data format as "channels_last",but it is not work,can you tell me what's wrong?

Thank you for sharing the code. I have been trying to apply your code a data set. However, running the code gives the following error message:
exception_prefix='input')
File "D:\Program Files\Python\Python35\lib\site-packages\keras\engine\training_utils.py", line 138, in standardize_input_data
str(data_shape))
ValueError: Error when checking input: expected conv2d_1_input to have shape (256, 256, 1) but got array with shape (1, 256, 256)

hey,
i might do something wrong which i cant understand, 2 problems appears when simply running the example and i would like to solve them.

1. running build_model.py results in this error:
   Traceback (most recent call last):
   File "build_model.py", line 123, in
   autoencoder.add(Reshape((n_labels, img_h * img_w)))
   File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 455, in add
   output_tensor = layer(self.outputs[0])
   File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 559, in call
   output_shape = self.compute_output_shape(input_shape)
   File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 385, in compute_output_shape
   input_shape[1:], self.target_shape)
   File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 379, in _fix_unknown_dimension
   raise ValueError(msg)
   ValueError: total size of new array must be unchanged

2. running SegNet.py results with this error:
   Traceback (most recent call last):
   File "SegNet.py", line 105, in
   autoencoder = models.model_from_json(model_file.read())
   File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 325, in model_from_json
   return layer_module.deserialize(config, custom_objects=custom_objects)
   File "/usr/local/lib/python2.7/dist-packages/keras/layers/init.py", line 46, in deserialize
   printable_module_name='layer')
   File "/usr/local/lib/python2.7/dist-packages/keras/utils/generic_utils.py", line 141, in deserialize_keras_object
   return cls.from_config(config['config'])
   File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 1190, in from_config
   model.add(layer)
   File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 455, in add
   output_tensor = layer(self.outputs[0])
   File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 554, in call
   output = self.call(inputs, **kwargs)
   File "/usr/local/lib/python2.7/dist-packages/keras/layers/normalization.py", line 140, in call
   epsilon=self.epsilon)
   File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 572, in normalize_batch_in_training
   x, gamma, beta, reduction_axes, epsilon)
   File "/usr/local/lib/python2.7/dist-packages/theano/tensor/nnet/bn.py", line 220, in batch_normalization_train
   raise ValueError("epsilon must be at least 1e-5, got %s" % str(epsilon))
   ValueError: epsilon must be at least 1e-5, got 9.99999997475e-07

Hello. I trained my dataset with your segnet implementation.
I used DePool2D custom layer when I trained my model.
Training was done smoothly, and it shows nice results.

However when I tried to reload the model it shows this error and I can't figure out what the problem is. Do you have any tips?

Traceback (most recent call last):
  File "infer_test.py", line 188, in <module>
    main()
  File "infer_test.py", line 183, in main
    join(model_root, "infer_results", splitext(snapshot)[0]))
  File "infer_test.py", line 91, in infer_data
    model = load_model(snapshot_path, custom_objects = {'DePool2D' : DePool2D(MaxPooling2D)})
  File "/usr/lib/python3.6/site-packages/keras/models.py", line 239, in load_model
    model = model_from_config(model_config, custom_objects=custom_objects)
  File "/usr/lib/python3.6/site-packages/keras/models.py", line 313, in model_from_config
    return layer_module.deserialize(config, custom_objects=custom_objects)
  File "/usr/lib/python3.6/site-packages/keras/layers/__init__.py", line 54, in deserialize
    printable_module_name='layer')
  File "/usr/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 139, in deserialize_keras_object
    list(custom_objects.items())))
  File "/usr/lib/python3.6/site-packages/keras/engine/topology.py", line 2487, in from_config
    process_layer(layer_data)
  File "/usr/lib/python3.6/site-packages/keras/engine/topology.py", line 2473, in process_layer
    custom_objects=custom_objects)
  File "/usr/lib/python3.6/site-packages/keras/layers/__init__.py", line 54, in deserialize
    printable_module_name='layer')
  File "/usr/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 141, in deserialize_keras_object
    return cls.from_config(config['config'])
  File "/usr/lib/python3.6/site-packages/keras/engine/topology.py", line 1252, in from_config
    return cls(**config)
TypeError: __init__() missing 1 required positional argument: 'pool2d_layer'

I load model with this codes

from build_segnet import DePool2D
...
model = load_model(snapshot_path, custom_objects = {'DePool2D' : DePool2D()})