imlab-uiip / keras-segnet Goto Github PK
View Code? Open in Web Editor NEWSegNet model implemented using keras framework
License: MIT License
SegNet model implemented using keras framework
License: MIT License
I am trying to convert the build.py to python 3 and keras version 2.1.4. I came a long way but I have some trouble with (hopefully) the last step. Below I posted the code I have now but what I need to do yet is the final reshaping. I receive the error ValueError: total size of new array must be unchanged
which according to this answer I got this error because the shape of my output is not corresponding to what I want to reshape it to. However my height and width are both factors of 32 so I do not understand the problem. The final layer of the network (a batch normalization) returns a shape of (Bat (None, 0, 256, 2)
. According to the code I found I have to reshape this using
autoencoder.add(Reshape((n_labels, img_h*img_w))
which would be (2, (256 * 256))
. This sounds incorrect the 256 * 256 part is very large, but I don't fully understand the reshape function.
I saw similair questions in other threads but there the problem seems to be that tensorflow was used instead of theano. I am using Theano as backend.
Can somebody shed some light on what this reshaping does and maybe suggest to what dimensions I need to reshape?
This is the entire model I'm using:
from keras import models
from keras.layers.core import Activation, Reshape, Permute
from keras.layers.convolutional import Conv2D, MaxPooling2D, UpSampling2D
from keras.layers.normalization import BatchNormalization
import json
img_w = 256
img_h = 256
n_labels = 2
kernel = 3
encoding_layers = [
Conv2D(64, (kernel,kernel), padding='same', input_shape=(1, img_h, img_w)),
BatchNormalization(),
Activation('relu'),
Conv2D(64, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
MaxPooling2D(),
Conv2D(128, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(128, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
MaxPooling2D(),
Conv2D(256, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(256, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(256, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
MaxPooling2D(),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
MaxPooling2D(),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
MaxPooling2D(),
]
autoencoder = models.Sequential()
autoencoder.encoding_layers = encoding_layers
for l in autoencoder.encoding_layers:
autoencoder.add(l)
decoding_layers = [
UpSampling2D(),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
UpSampling2D(),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(512, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(256, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
UpSampling2D(),
Conv2D(256, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(256, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(128, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
UpSampling2D(),
Conv2D(128, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(64, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
UpSampling2D(),
Conv2D(64, (kernel,kernel), padding='same'),
BatchNormalization(),
Activation('relu'),
Conv2D(n_labels, (1, 1), padding='valid'),
BatchNormalization(),
]
autoencoder.decoding_layers = decoding_layers
for l in autoencoder.decoding_layers:
autoencoder.add(l)
autoencoder.add(Reshape((n_labels, img_h * img_w)))
autoencoder.add(Permute((2, 1)))
autoencoder.add(Activation('softmax'))
with open('model_5l.json', 'w') as outfile:
outfile.write(json.dumps(json.loads(autoencoder.to_json()), indent=2))
Got the following error when I run the following command: python SegNet.py
Using Theano backend.
Using gpu device 0: GeForce GTX 1070 (CNMeM is disabled, cuDNN 5105)
/usr/local/lib/python2.7/dist-packages/theano/sandbox/cuda/__init__.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
warnings.warn(warn)
Compiled: OK
Traceback (most recent call last):
File "SegNet.py", line 118, in <module>
autoencoder.load_weights('model_5l_weight_ep50.hdf5')
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 2695, in load_weights
f = h5py.File(filepath, mode='r')
File "/usr/lib/python2.7/dist-packages/h5py/_hl/files.py", line 272, in __init__
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/usr/lib/python2.7/dist-packages/h5py/_hl/files.py", line 92, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/build/h5py-nQFNYZ/h5py-2.6.0/h5py/_objects.c:2577)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/build/h5py-nQFNYZ/h5py-2.6.0/h5py/_objects.c:2536)
File "h5py/h5f.pyx", line 76, in h5py.h5f.open (/build/h5py-nQFNYZ/h5py-2.6.0/h5py/h5f.c:1811)
IOError: Unable to open file (File signature not found)
hi,i am running build_model.py,it raises ValueError: Negative dimension size caused by subtracting 2 from 1 for 'max_pooling2d_2/MaxPool' (op: 'MaxPool') with input shapes: [?,1,1,256].i have tried to set data format as "channels_last",but it is not work,can you tell me what's wrong?
Hi,
as far as i understand, this implementation does not use the index-based upsampling, but uses the Upsample2D Keras layer. Do you have any ideas on how to implement the index passing and how much of the impact this should have?
Thanks!
Kind regards,
Tanja
Hi,
Your code is of great value for people who are new to DL in Keras and Segnet, and I really appreciate your work. I found you use function prep_data(mode) to make label for the training and testing dataset, but there's not any example illustrating what's the .CSV file should look like, could you just kindly give an example of your training dataset and the CSV file corresponding to it?
Thank you,
Tony
Hi, with respect to this segnet model, we can see that in each building block of encoder part, there are always two convolutional layers following each other before a maxpooling layer. For instance,
Convolution2D(64, kernel, kernel, border_mode='same', input_shape=(1, img_h, img_w)),
BatchNormalization(),
Activation('relu'),
Convolution2D(64, kernel, kernel, border_mode='same'),
BatchNormalization(),
Activation('relu'),
MaxPooling2D(),
I am not very clear about the real logic of having two convolutional layers following each other. Why not just having a combined convolutional layer. What's the strength of having this kind of design shown in Segnet architecture.
Ran into this issue when the following command is run: python SegNet.py
/usr/lib/python2.7/dist-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
Using Theano backend.
Using gpu device 0: GeForce GTX 1070 (CNMeM is disabled, cuDNN 5105)
/usr/local/lib/python2.7/dist-packages/theano/sandbox/cuda/__init__.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
warnings.warn(warn)
Traceback (most recent call last):
File "SegNet.py", line 105, in <module>
autoencoder = models.model_from_json(model_file.read())
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 210, in model_from_json
return layer_from_config(config, custom_objects=custom_objects)
File "/usr/local/lib/python2.7/dist-packages/keras/utils/layer_utils.py", line 40, in layer_from_config
return layer_class.from_config(config['config'])
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 1080, in from_config
model.add(layer)
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 327, in add
output_tensor = layer(self.outputs[0])
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 569, in __call__
self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 632, in add_inbound_node
Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 168, in create_node
output_shapes = to_list(outbound_layer.get_output_shape_for(input_shapes[0]))
File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 336, in get_output_shape_for
self.target_shape)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 330, in _fix_unknown_dimension
raise ValueError(msg)
ValueError: total size of new array must be unchanged
Hi, thanks for sharing the code.
I have two questions:
1 tensorflow backend. I use tensorflow back to train the model, But the predicted output is poor.
2 how to use your index-based-upsampling code? If I use tensorflow backend, how do I modify it?
Thank you.
I am trying to run this in windows python 3 with tensor`flow backend(somehow my theano backend fail to utilize gpu). i converted print syntax in the ipython notebook and tried to run the script, it went well, but when i reached to "Loading and compiling model built with 'build_model.py'", received the following error.
May I know what needs to be changed, based on this error. Also, what would be your suggestion if I would like to run the ipython using tensorflow backend with python 3
`thank you in advance for looking into this issue.
Hi, I get the following error when loading the json model. I have tried in both Keras vs 1 and 2 and I run with Theano as backend.
File "", line 1, in
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/models.py", line 345, in model_from_json
return layer_module.deserialize(config, custom_objects=custom_objects)
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/layers/init.py", line 54, in deserialize
printable_module_name='layer')
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/utils/generic_utils.py", line 139, in deserialize_keras_object
list(custom_objects.items())))
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/models.py", line 1214, in from_config
model.add(layer)
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/models.py", line 475, in add
output_tensor = layer(self.outputs[0])
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/engine/topology.py", line 621, in call
output_shape = self.compute_output_shape(input_shape)
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/layers/core.py", line 376, in compute_output_shape
input_shape[1:], self.target_shape)
File "/chuan/virtualEnvironments/Keras/keras-env/local/lib/python2.7/site-packages/keras/layers/core.py", line 370, in _fix_unknown_dimension
raise ValueError(msg)
ValueError: total size of new array must be unchanged
Thank you for sharing the code. I have been trying to apply your code a data set. However, running the code gives the following error message:
exception_prefix='input')
File "D:\Program Files\Python\Python35\lib\site-packages\keras\engine\training_utils.py", line 138, in standardize_input_data
str(data_shape))
ValueError: Error when checking input: expected conv2d_1_input to have shape (256, 256, 1) but got array with shape (1, 256, 256)
Hi,
I have generated the training data with corresponding ground truths and the required ".csv" file. How can I train it from scratch using my data ? Can you please elaborate the steps ?
Thanks !
hey,
i might do something wrong which i cant understand, 2 problems appears when simply running the example and i would like to solve them.
running build_model.py results in this error:
Traceback (most recent call last):
File "build_model.py", line 123, in
autoencoder.add(Reshape((n_labels, img_h * img_w)))
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 455, in add
output_tensor = layer(self.outputs[0])
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 559, in call
output_shape = self.compute_output_shape(input_shape)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 385, in compute_output_shape
input_shape[1:], self.target_shape)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 379, in _fix_unknown_dimension
raise ValueError(msg)
ValueError: total size of new array must be unchanged
running SegNet.py results with this error:
Traceback (most recent call last):
File "SegNet.py", line 105, in
autoencoder = models.model_from_json(model_file.read())
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 325, in model_from_json
return layer_module.deserialize(config, custom_objects=custom_objects)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/init.py", line 46, in deserialize
printable_module_name='layer')
File "/usr/local/lib/python2.7/dist-packages/keras/utils/generic_utils.py", line 141, in deserialize_keras_object
return cls.from_config(config['config'])
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 1190, in from_config
model.add(layer)
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 455, in add
output_tensor = layer(self.outputs[0])
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 554, in call
output = self.call(inputs, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/normalization.py", line 140, in call
epsilon=self.epsilon)
File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 572, in normalize_batch_in_training
x, gamma, beta, reduction_axes, epsilon)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/nnet/bn.py", line 220, in batch_normalization_train
raise ValueError("epsilon must be at least 1e-5, got %s" % str(epsilon))
ValueError: epsilon must be at least 1e-5, got 9.99999997475e-07
Seem in the last layers you reshape all to [2,65536] (assuming this is row x col) and 256*256 is 65536, so it's just image with 2 'channels' (assuming background/foreground) reshaped to flat vector. So in general case it will be [n_classes, h*w]
. And than softmax is putted on top.
https://github.com/imlab-uiip/keras-segnet/blob/master/SegNet.py#L19
https://github.com/imlab-uiip/keras-segnet/blob/master/model_5l.json#L1266
Also here you use categorical_crossentropy
loss.
https://github.com/imlab-uiip/keras-segnet/blob/master/SegNet.py#L108
My confusion is how softmax is applied and how loss is computed, as I understand it should be like:
compute softmax for each [2,i] (and not to [i,65536]) subvector separately of matrix and loss should be computed for each pixel, so as I understand Keras just do it all in some vectorized form?
So what is the rules to pass matrix/tensor to softmax+categorical_crossentropy
? what shape shoul it be? row represent dimensions and cols represent pixels?
Documentation is not rich about this topics:
https://keras.io/activations/#softmax
https://keras.io/losses/#categorical_crossentropy
BTW what is profit of store model as .json ? I find this solution less readable.
I tried pulling the .hdf5 file using git lfs, but I am getting this error.
batch response: This repository is over its data quota. Purchase more data packs to restore access.
Is there any other way to download the file?
Hi
Thank you for sharing the code. I have been trying to apply your code a data set, which has image size of 128*128. However, running the code gives the following error message,
ValueError: Cannot feed value of shape (32, 1, 128, 128) for Tensor 'activation_26_target:0', which has shape '(?, ?, ?)'
Would you like to share any thought on the possible reason? Is that because the current implementation can only accept images with 256. If the image is of other sizes, we have to change the architecture. Thanks.
The following is the whole traceback
File "/development/tfw/lib/python3.4/site-packages/Keras-1.0.3-py3.4.egg/keras/models.py", line 409, in fit
sample_weight=sample_weight)
File "/development/tfw/lib/python3.4/site-packages/Keras-1.0.3-py3.4.egg/keras/engine/training.py", line 1052, in fit
callback_metrics=callback_metrics)
File "/development/tfw/lib/python3.4/site-packages/Keras-1.0.3-py3.4.egg/keras/engine/training.py", line 790, in _fit_loop
outs = f(ins_batch)
File "/development/tfw/lib/python3.4/site-packages/Keras-1.0.3-py3.4.egg/keras/backend/tensorflow_backend.py", line 659, in __call__
updated = session.run(self.outputs + self.updates, feed_dict=feed_dict)
File "/development/tfw/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 340, in run
run_metadata_ptr)
File "/development/tfw/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 553, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (32, 1, 128, 128) for Tensor 'activation_26_target:0', which has shape '(?, ?, ?)'
Hello. I trained my dataset with your segnet implementation.
I used DePool2D custom layer when I trained my model.
Training was done smoothly, and it shows nice results.
However when I tried to reload the model it shows this error and I can't figure out what the problem is. Do you have any tips?
Traceback (most recent call last):
File "infer_test.py", line 188, in <module>
main()
File "infer_test.py", line 183, in main
join(model_root, "infer_results", splitext(snapshot)[0]))
File "infer_test.py", line 91, in infer_data
model = load_model(snapshot_path, custom_objects = {'DePool2D' : DePool2D(MaxPooling2D)})
File "/usr/lib/python3.6/site-packages/keras/models.py", line 239, in load_model
model = model_from_config(model_config, custom_objects=custom_objects)
File "/usr/lib/python3.6/site-packages/keras/models.py", line 313, in model_from_config
return layer_module.deserialize(config, custom_objects=custom_objects)
File "/usr/lib/python3.6/site-packages/keras/layers/__init__.py", line 54, in deserialize
printable_module_name='layer')
File "/usr/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 139, in deserialize_keras_object
list(custom_objects.items())))
File "/usr/lib/python3.6/site-packages/keras/engine/topology.py", line 2487, in from_config
process_layer(layer_data)
File "/usr/lib/python3.6/site-packages/keras/engine/topology.py", line 2473, in process_layer
custom_objects=custom_objects)
File "/usr/lib/python3.6/site-packages/keras/layers/__init__.py", line 54, in deserialize
printable_module_name='layer')
File "/usr/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 141, in deserialize_keras_object
return cls.from_config(config['config'])
File "/usr/lib/python3.6/site-packages/keras/engine/topology.py", line 1252, in from_config
return cls(**config)
TypeError: __init__() missing 1 required positional argument: 'pool2d_layer'
I load model with this codes
from build_segnet import DePool2D
...
model = load_model(snapshot_path, custom_objects = {'DePool2D' : DePool2D()})
Hi @eclique,
Could you tell us what versions of python packages you are using?
Many thanks
Ian
Dear all,
I wish to apply Segnet Model for image segmentation task. These are medical images. I want to initialize the weights for Segnet model using Camvid dataset. What changes can I do to the current model to initialize the weights? I saw some thread https://github.com/legokichi/keras-segnet/blob/master/model_segnet.py, where they initialize weights using VGG16 pre-trained on Camvid dataset. But I couldn't really run that script (some error).
Any help would be appreciated.
Thank you
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.