aurora95 / keras-fcn Goto Github PK

Hi @aurora95 and @athundt, When I run the train.py file, it raises the following error.
Could you please suggest me some way to solve this issue.

File "train.py", line 213, in
label_suffix=label_suffix, ignore_label=ignore_label, label_cval=label_cval)
File "train.py", line 83, in train
classes=classes)
ValueError: axes don't match array

if dataset == 'VOC2012_BERKELEY':
train_file_path = os.path.expanduser('/tf/keras-fcn/datasets/VOC2012/ImageSets/Segmentation/train.txt')
val_file_path = os.path.expanduser('/tf/keras-fcn/datasets/VOC2012/ImageSets/Segmentation/val.txt')
data_dir = os.path.expanduser('/tf/keras-fcn/datasets/VOC2012/JPEGImages')
label_dir = os.path.expanduser('/tf/keras-fcn/datasets/VOC2012/SegmentationClass')
data_suffix='.jpg'
label_suffix='.png'
classes = 21

Hi there.

Nice job on the repo! I am trying to train the FCN_Vgg16_32s model, and I am having issue with the loss being "nan".

I followed the steps to set up the VOC dataset and transfer VGG weights for the FCN model.

The only code changes I made was in train.py was setting model_name='FCN_Vgg16_32s' and the loss after a few steps, results to nan.

Here is a screenshot of my terminal output.

Please advise on how I should go about debugging the loss function, and if you need additional information.

Hi aurora95,

Thanks for sharing the code.

I am planning to adopt your code to training my own data set. The training images are of size 500*700, and each of which is black-and-white, i.e., with only one single channel. What kind of modifications have to be made to make your code fitful for my data set? Thanks a lot.

Wenouyang

The function should be updated to support Keras2.0 or the output shape can't be rightly reached.
Can be done by simply change the name of the function

I'm running with the current master and I'm not seeing the performance described in #4, perhaps something is up with my converted dataset?

11127/11127 [==============================] - 7896s - loss: 1.3891 - sparse_accuracy_ignoring_last_label: 0.6165
lr: 0.009964
Epoch 2/250
11127/11127 [==============================] - 7972s - loss: 1.0751 - sparse_accuracy_ignoring_last_label: 0.6326
lr: 0.009928
Epoch 3/250
11127/11127 [==============================] - 7937s - loss: 1.0529 - sparse_accuracy_ignoring_last_label: 0.6385
lr: 0.009892
Epoch 4/250
11127/11127 [==============================] - 7878s - loss: 1.0487 - sparse_accuracy_ignoring_last_label: 0.6407
lr: 0.009856
Epoch 5/250
11127/11127 [==============================] - 7915s - loss: 1.0411 - sparse_accuracy_ignoring_last_label: 0.6434
lr: 0.009820
Epoch 6/250
11127/11127 [==============================] - 7849s - loss: 1.0374 - sparse_accuracy_ignoring_last_label: 0.6447
lr: 0.009784
Epoch 7/250
11127/11127 [==============================] - 7843s - loss: 1.0358 - sparse_accuracy_ignoring_last_label: 0.6448
lr: 0.009748
Epoch 8/250
 6808/11127 [=================>............] - ETA: 3041s - loss: 1.0342 - sparse_accuracy_ignoring_last_label: 0.6447

Also training is taking a lot longer than I imagined with around 2 hours per epoch, is that typical with the full 11k images from pascal voc + the berkeley dataset? I'm running on a GTX1080 with a batch size of 16 and the files are stored on an HDD, not an SSD, though theoretically linux does some caching for this sort of thing and it could all fit in my 48GB of system ram.

I meet the error in epoch 1 with VOC2012

lr: 0.010000
Epoch 1/250
Traceback (most recent call last):
File "D:\Python\Python36\lib\site-packages\keras\utils\data_utils.py", line 513, in get
inputs = self.queue.get(block=True).get()
File "D:\Python\Python36\lib\multiprocessing\pool.py", line 644, in get
raise self._value
File "D:\Python\Python36\lib\multiprocessing\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "D:\Python\Python36\lib\site-packages\keras\utils\data_utils.py", line 379, in get_index
return ds[i]
File "D:\Python\Python36\lib\site-packages\keras\preprocessing\image.py", line 759, in getitem
return self._get_batches_of_transformed_samples(index_array)
File "D:\Python\Python36\lib\site-packages\keras\preprocessing\image.py", line 805, in _get_batches_of_transformed_samples
raise NotImplementedError
NotImplementedError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:/demo/pycharmWorksapce/Keras-FCN-master/train.py", line 234, in
label_suffix=label_suffix, ignore_label=ignore_label, label_cval=label_cval)
File "D:/demo/pycharmWorksapce/Keras-FCN-master/train.py", line 163, in train
class_weight=class_weight
File "D:\Python\Python36\lib\site-packages\keras\legacy\interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "D:\Python\Python36\lib\site-packages\keras\engine\training.py", line 2046, in fit_generator
generator_output = next(output_generator)
File "D:\Python\Python36\lib\site-packages\keras\utils\data_utils.py", line 518, in get
raise StopIteration(e)
StopIteration

Process finished with exit code 1

Hello aurora95

I am trying to use SegDataGenerator. However, it is too slow.
I am running the code on TitanX and 64 GB memory.
the output of my network is (None, 50176, 2). Therefore, I have modified SegDataGenerator by making the shape of y equals to (None, 50176, 2 ). In addition, I am using categorical cross entropy.
batch_size =64
My question is: why loading the training images is very slow?
For example, one epoch needs 151051s.

Thank you

Would you be interested in submitting or giving permission to submit SegDataGenerator to keras?

It would first go through keras-contrib as per the Keras instructions for pull requests.

I wouldn't mind doing the integration, I can make sure credit is properly given as well.

The code raise this error when I try "python train.py "
but I check the egg file and found the "from keras.backend.tensorflow_backend import _postprocess_conv3d_output" in line10. so I try to use print(dir()) to show what this module contains.
but I can only see _preprocess_conv2d_input', '_preprocess_conv3d_input', '_preprocess_padding',
there is no function named _postprocess_conv3d_output
How can it be?

If i skip MS COCO step, the error will occur or only the performance will reduced??

When I run "git clone [email protected]:ahundt/keras-contrib.git -b densenet-atrous"
there is a mistake :

Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

So I find another keras- contrib ： git clone https://www.github.com/farizrahman4u/keras-contrib.git

however when i run train.py there is a mistake:
Traceback (most recent call last):
File "train.py", line 15, in
from models import *
File "/home/deeplearning/src/Keras-FCN/models.py", line 6, in
from keras_contrib.applications import densenet
File "build/bdist.linux-x86_64/egg/keras_contrib/init.py", line 13, in
File "build/bdist.linux-x86_64/egg/keras_contrib/optimizers/init.py", line 1, in
File "build/bdist.linux-x86_64/egg/keras_contrib/optimizers/ftml.py", line 8, in
File "build/bdist.linux-x86_64/egg/keras_contrib/optimizers/ftml.py", line 34, in FTML
AttributeError: 'module' object has no attribute 'legacy_get_updates_support'

When I run python train.py, it goes wrong.
I wander where is the h5 format file.
Could you please help me with this problem

Traceback (most recent call last):
  File "train.py", line 120, in <module>
    data_dir, label_dir, target_size=target_size, batchnorm_momentum=batchnorm_momentum, resume_training=resume_training)
  File "train.py", line 49, in train
    model = globals()[model_name](weight_decay=weight_decay, input_shape=input_shape, batch_momentum=batchnorm_momentum)
  File "/home/cv/keras-fcn/Keras-FCN-master/models.py", line 209, in AtrousFCN_Resnet50_16s
    model.load_weights(weights_path, by_name=True)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 2702, in load_weights
    f = h5py.File(filepath, mode='r')
  File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 272, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 92, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2684)
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2642)
  File "h5py/h5f.pyx", line 76, in h5py.h5f.open (/tmp/pip-4rPeHA-build/h5py/h5f.c:1930)
IOError: Unable to open file (Unable to open file: name = '/home/cv/.keras/models/fcn_resnet50_weights_tf_dim_ordering_tf_kernels.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

Thanks a lot for this wonderful code!

I have a question about the model. I found in the model, you implemented a BilinearUpSampling2D class by yourself. How about if I replace it as the function "UpSampling2D" in Keras? As I am trying to convert this model into coreml and run it on the iphone. The original function in the model is not supported by coreml. But the UpSampling2D is supported.

Thanks in advanced!

I've run train.py with all the default setting on PASCAL VOC dataset with AtrousFCN_Resnet50_16s with a epoch number of 250. the result is not satisfactory with a Mean IOU of 0.24 and the segmented pictures generated by the evaluate.py are barely recognized. I've run both train.py and evaluate.py on a nvidia TITAN X with 12G memory. What is the problem? I've also run train.py with other nets like VGG16 or Resnet50_32s. The results are even worse.

Hi,
I've been working on implementing a FCN and use it on the BRATS data set, and I'd like to use your implementation of resnet and see how it works. I was hoping I could get some help for some questions to train.
The data set of BRATS consists of:
- (155x240x240)x4. (155x240x240) is the image size and x4 is the modalities (instead of the normal 3 RGB channels).
- Groundtruth: segmented image with dimension: 155x240x240.

My question is how do I adapt your Resnet50 FCN implementation for my data?

I've read: https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf , and as I've understood I do only need to upsample the output, but I'm uncertain if this is correct.

Best Regards, Ssica

您好，用您的代码跑通了pascal voc数据集，现在我想训练自己得数据，数据集标注时候类别和颜色是怎么对应得呀？

I am wondering how to use sparse_categorical_crossentropy loss function and how to set the loss_shape parameter properly?

My model is defined as follows:

input_shape = [128, 128]
nb_labels=3
def get_model(input_shape=input_shape, nb_labels=3):

    nb_rows, nb_cols = input_shape
    inputs = Input((nb_rows, nb_cols, 3))
    ...
    ...
    ...
    conv10 = Conv2D(nb_labels, (1, 1), activation='linear')(conv9_2)
    x = Reshape((nb_rows * nb_cols, nb_labels))(conv10)
    x = Activation('softmax')(x)
   outputs = x
   model = Model(inputs=inputs, outputs=outputs)
   return model

In the main function, I set:

loss_fn = binary_crossentropy_with_logits
#loss_shape = (target_size[0] * target_size[1] * nb_labels,) # does not work!?
loss_shape = (target_size[0] * target_size[1], 1) # does work

Is that setting correct? Why does loss_shape = (target_size[0] * target_size[1] * nb_labels,) does not work? Usually, when I use the sparse_categorical_crossentropy loss, I do the following for converting the masks to one-hot-encoding: y_train = y_train.reshape((-1, 1)) and I was not sure where to add this in the SegDataGenerator. Any help is appreciated.

https://github.com/aurora95/Keras-FCN/blame/master/utils/SegDataGenerator.py#L486

@aurora95 @ahundt I would like to help with this, since I need it for my project. Any thoughts/resources on how to approach this? with or without calibr image?

EDIT: Found this solution, which seems simple enough to implement
http://www.tannerhelland.com/4743/simple-algorithm-correcting-lens-distortion/

It seems models.py assumes a weights file exists. With the repository as is it doesn't, did you initialize with imagenet weights or something or simply comment those lines the first time you run?

Hello,

Thank you for providing your code. I have been able to successfully training and evaluate data using the VOC2011 and VOC 2012 datasets with the model "AtrousFCN_Resnet50_16s".

I am working on training using my own data and the model "AtrousFCN_Resnet50_16s". I have a series of rgb .jpgs which I have segmented similarly to the VOC2011 and VOC2012 datasets. I am training for 3 classes (G1, G2, and no data) which are represented by 3 colors (purple, green, black), the first of two do not match the colors of the VOC models and are new categories. I should add that following the VOC data, I've also included a VOID category that is an outline around my objects of interest.

I have generated code to match the format of the VOC2012 dataset call in train.py. In this duplication, I have also changed the number of classes to 3:

if dataset == 'Grapes':
train_file_path = os.path.expanduser('/path/to/my/data/train.txt')
val_file_path = os.path.expanduser('/path/to/my/data/val.txt')
data_dir = os.path.expanduser('/path/to/my/data/JPEGImages')
label_dir = os.path.expanduser('/path/to/my/data/SegmentationObject')
data_suffix='.jpg'
label_suffix='.jpg'
classes = 3

When I run my code, I get an error in the SegDataGenerator which reads:
ValueError: could not broadcast input array from shape (350,350,3) into shape (350,350,1)

I've tracked back and the error occurs here in the code:

        if self.loss_shape is not None:
            y = np.reshape(y, self.loss_shape)

        batch_x[i] = x
        batch_y[i] = y

I noticed that in your COCO implementation, you added loss_shape and I added "loss_shape = (target_size[0] * target_size[1] * classes,)" to the configuration, but now I get a new error:

ValueError: Cannot feed value of shape (1, 367500) for Tensor 'bilinear_up_sampling2d_1_target:0', which has shape '(?, ?, ?, ?)'

Do you have any suggestions for what I need to do so that I can train on my own data? I might add that my input data has fewer rows and columns than the VOC data by about 50%.

Thank you for your response.

I think there's a bug in the cropping functions. In particular, I believe the correct order should be (h, w) instead of (w, h).

In the constructor, self.row_index (height) is set to 1 and self.col_index (width) to 2. However, in the crop functions, e.g. pair_random_crop, the width dimension is cropped as the first dimension.

Maybe I'm missing something?

I couldn't find the implementation of BilinearUpsampling2D layer in your implementation, could you direct me to it?

I have implemented Mean IoU metrics, based on calculate_iou in evaluate.py.
Could someone check it out

def sparse_iou_ignoring_last_label(y_true, y_pred):
     # mIoU
    nb_classes = K.int_shape(y_pred)[-1]
    y_pred = K.reshape(y_pred, (-1, nb_classes))
    pred = K.argmax(y_pred, axis=1)

    gt = tf.reshape(y_true, [-1,])
    weights = tf.cast(tf.less_equal(gt, nb_classes - 1), tf.int32) # Ignoring all labels greater than or equal to n_classes.

    confusion_matrix = tf.confusion_matrix(gt, pred, num_classes=nb_classes, weights=weights, dtype=tf.float32)

    I = tf.diag_part(confusion_matrix)
    U = tf.reduce_sum(confusion_matrix, axis=0) + tf.reduce_sum(confusion_matrix, axis=1) - I

    IoU = I / (U + tf.convert_to_tensor(K.epsilon(), dtype=tf.float32))
    return tf.reduce_mean(IoU) 

If it is correct, I want to extend it when batch_size > 1

Hello, thank you for your code.
When I try to run the train.py to do a demo, but I get some error below.

lr: 0.010000
Epoch 1/250
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/keras/utils/data_utils.py", line 513, in get
inputs = self.queue.get(block=True).get()
File "/usr/lib/python3.4/multiprocessing/pool.py", line 599, in get
raise self._value
File "/usr/lib/python3.4/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/local/lib/python3.4/dist-packages/keras/utils/data_utils.py", line 379, in get_index
return ds[i]
File "/usr/local/lib/python3.4/dist-packages/keras/preprocessing/image.py", line 759, in getitem
return self._get_batches_of_transformed_samples(index_array)
File "/usr/local/lib/python3.4/dist-packages/keras/preprocessing/image.py", line 805, in _get_batches_of_transformed_samples
raise NotImplementedError
NotImplementedError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/s405/pengcheng/project/Keras-FCN/train.py", line 231, in
label_suffix=label_suffix, ignore_label=ignore_label, label_cval=label_cval)
File "/home/s405/pengcheng/project/Keras-FCN/train.py", line 163, in train
class_weight=class_weight
File "/usr/local/lib/python3.4/dist-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/keras/engine/training.py", line 2046, in fit_generator
generator_output = next(output_generator)
File "/usr/local/lib/python3.4/dist-packages/keras/utils/data_utils.py", line 518, in get
raise StopIteration(e)
StopIteration
I don't know how to fix this problem, can you give me some suggestions?

HI, I'm new in this field, I use your code to do segmentation. I confuse about the loss func.
For example I have two classes. [0] represents background class and [1] represent another class. My label '.png' has the shape like [0 0 0 0 1 1 0 0...] (2dim). Now I use softmax_sparse_crossentropy_ignoring_last_label(y_true, y_pred) as my loss function.

It says

ignoring_last_label

I'm wondering which label it will ignore? class[0], class[1], or..another?
I trace the code and find ..it seems still have 2 classes shape.

my y_pred, the out put of my model has the shape (num,224,224,2)
And I want to calculate 2 classes loss.

Many Thanks!

Hello,

How can I fine-tune the model on a custom small dataset using a pretrained model?

I am new to Keras,and github.And I do not know how to run this code.Using this code to image segment.I have training set and labels.it is a lot of jpg image.How can I use them.Thank you very much.

Is the .h5 weights file downloaded from somewhere else? Or did you train your weights?

For example in model.py

weights_path = os.path.expanduser(os.path.join('~', '.keras/models/fcn_vgg16_weights_tf_dim_ordering_tf_kernels.h5'))
model.load_weights(weights_path, by_name=True)
return model

Based on the file path .keras/, it looks like a weights file come from Keras. But I am not aware of that keras provides fcn weights.

can someone share pre-trained model (AtrousFCN_Resnet50_16s)?
Thanks

Thank you for your code.
I've already download voc2012 dataset. But I didn't find this file“combined_imageset_train.txt”,Similarly, I didn't find “combined_imageset_val.txt”,If you are free, please tell me

Any chance you could give this a license? One good possible option would be the MIT license https://tldrlegal.com/license/mit-license, this is what Keras itself uses. In fact they might be interested in a pull request of this code to the official keras-contrib repository. https://github.com/farizrahman4u/keras-contrib

Also what kind of IOU results did you get with the models you define in this repository?

Thanks!

Hello, thank you for your code.
When I try to run the train.py to do a demo, but I get a "TypeError: pad_width must be of integral type.", and it points to the "Keras-FCN\utils\SegDataGenerator.py", line 238, in next
x = np.lib.pad(x, ((pad_h / 2, pad_h - pad_h / 2), (pad_w / 2, pad_w - pad_w / 2), (0, 0)), 'constant', constant_values=0.)". I don't know how to fix this problem, can you give me some suggestions?

I have 300x300 single channel images. I am trying a segmentation problem. I have two classes - object1 and object2, and the rest is background. Therefore, my output is of size 300x300x2. 1st channel has 0s representing background and object2, and 1s representing object1. 2nd channel has 0s representing background and object1, and 1s representing object2. What changes are required in the "softmax_sparse_crossentropy_ignoring_last_label" function in order for me to be able to use it, because your loss function involves finding the one hot vector from the output obtained, and I believe my output is already in that form.
Is this part of the code required for my application?

y_true = K.one_hot(tf.to_int32(K.flatten(y_true)), K.int_shape(y_pred)[-1]+1)

unpacked = tf.unstack(y_true, axis=-1)

y_true = tf.stack(unpacked[:-1], axis=-1)

when I python train.py ,meet AttributeError: 'module' object has no attribute 'legacy_get_updates_support' why?

I tried using your code for image-segmenting. Size of my dataset was 68683.
So I modified your code according to my dataset.
The error occurs at BilinearUpSampling2D step. I'm not sure how to resolve it.
link to the code : (https://github.com/PRRvalli/Deep_learning-codes/blob/master/FCN-implementation%20on%20rtmi.ipynb).

I train the model on VOC2012 segmentation, using 250 epoch and 20 batch_size.
The training accu is 0.985~.(It should be the pixel accu, right?)
But the testing mIoU is just 0.56025, which has a big gap with the result in the paper.
What's your result?
And maybe the hyperparameter is not optimal?

[ 0.9077976 0.75288631 0.47242709 0.58043887 0.50712961 0.47588975
  0.72664924 0.68814923 0.66110743 0.20972129 0.53752346 0.31540707
  0.5731418 0.50105971 0.64968748 0.76978092 0.43357291 0.5765887
  0.32090376 0.61638412 0.49687383]
meanIOU: 0.560625
pixel acc: 0.898940

Hi,

after analysing the code and comparing the model image, I don't find the difference of the network structure between FCN_Resnet50_32s and AtrousFCN_Resnet50_16s. The only difference is that AtrousFCN_Resnet50_16s used atrous_identity_block instead of the original in the last two identity blocks. But I don't get essential difference between atrous_identity_block and the original one either. As far as I know, the main difference between xxxx_32s and xxx_16s is a skip connection from the 16s downsampling layer to the 32s one. Am I right? Thanks.

Hello,
please , may upgrade your code to python 3 and keras 2. (only train.py and sub files)
Thanks

After i trained the first epoch，i meet this error
File "/home/s405/pengcheng/project/Keras-FCN/train.py", line 232, in
label_suffix=label_suffix, ignore_label=ignore_label, label_cval=label_cval)
File "/home/s405/pengcheng/project/Keras-FCN/train.py", line 164, in train
class_weight=class_weight
File "/usr/local/lib/python3.4/dist-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/keras/engine/training.py", line 2082, in fit_generator
callbacks.on_epoch_end(epoch, epoch_logs)
File "/usr/local/lib/python3.4/dist-packages/keras/callbacks.py", line 77, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "/usr/local/lib/python3.4/dist-packages/keras/callbacks.py", line 751, in on_epoch_end
raise ValueError('If printing histograms, validation_data must be '
ValueError: If printing histograms, validation_data must be provided, and cannot be a generator.

I don't know how to fix this problem, can you give me some suggestions?

Hi,

First of all, I want to thank you for sharing your code. I use your code for my one of my projects. I made several updates according to my needs.
But I believe one of them critical for the rest of the community.
I normalized my samples between 0 and 1, then sample wise zero mean and normalization.

example code:
train_datagen = SegDataGenerator(rescale=1./255,
samplewise_center=True,
samplewise_std_normalization=True,..)

I realized that you call preprocess_input() function at line 284

batch_x = preprocess_input(batch_x)

This line causes an additional zero-center by mean (it assumes that we have a non standardized 0-255 ranged image):
x[:, :, :, 0] -= 103.939
x[:, :, :, 1] -= 116.779
x[:, :, :, 2] -= 123.68

since we already applied x = self.seg_data_generator.standardize(x) at line 258, we need to remove line 284. (or it should be done in standardize() function if needed )

Questions by @mptorr in keras-team/keras#6538 (comment)

I was going to try your SegDataGenerator however wanted to ask 2 things about it, as they may pertain to your request for features/suggestions:

[1] it appears it currently does not support pixelwise weighting to compensate for class imbalance. This would be an important feature to me, as most of my segmentation tasks will have disproportionately over/under-represented classes. Currently I balance classes using Keras' sample_weighting as temporal, however without data augmentation due to my issues above. Since the sample weighting matches each pixel in the image/mask pair, it would need to be appropriately transformed to match augmented images/masks. Let me know if I'm overlooking this feature in SegDataGenerator.

It isn't there yet unfortunately. Suggestions/pull requests are welcome.

[2] I'm a bit confused on how SegDataGenerator loads images. The comment in the class perhaps could be reworded (or have examples) for the most important arguments. I also didn't understand how to use this info: for a file name 2011_002920.jpg, each row should contain 2011_002920. Of course, this lack of understanding may reflect my own limitations, but just thought it could help development.

SegDataGenerator expects the data to be in pascal_voc format, since the images are in jpg and the labels are in png the authors decided to put neither in the dataset list txt files. If you walk through the process in the README.md you'll end up with a folder properly set up for running which makes things much more clear.

Hi guys, thanks for the effort. For a weird reason I can't install tf on my system so I have to use keras with theano. Was there a valid reason you implemented this tf dependent (and i should give up converting this while i haven't put much effort)?

I run train.py toward my own data_path
train_file_path = os.path.expanduser('/home/wang/Li_XC/KSFRCNN/data/VOCdevkit/VOC2012/ImageSets/Segmentation/train.txt') #Data/VOClarge/VOC2012/ImageSets/Segmentation
val_file_path = os.path.expanduser('/home/wang/Li_XC/KSFRCNN/data/VOCdevkit/VOC2012/ImageSets/Segmentation/val.txt')
data_dir = os.path.expanduser('/home/wang/Li_XC/KSFRCNN/data/VOCdevkit/VOC2012/JPEGImages')
label_dir = os.path.expanduser('/home/wang/Li_XC/KSFRCNN/data/VOCdevkit/VOC2012/SegmentationClass')
But I got a very very low IOU matrix when I run evaluate.py.
meanIOU: 0.014367
pixel acc: 0.124107
Could somebody helps?

Hi,

Where can I download the model weights? Forexample "fcn_resnet50_weights_tf_dim_ordering_tf_kernels.h5"
Thanks

I tried enabling the validation code in SegDataGenerator plus using the keras early stopping and ReduceLROnPlateau which I put in train.py, uncommenting the validation lines in my validation_callbacks branch forked from your repo.

Unfortunately it seems to fail with an error like the following:

  File "train.py", line 132, in <module>                                                                                                [1170/7924]
    data_dir, label_dir, target_size=target_size, batchnorm_momentum=batchnorm_momentum, resume_training=resume_training)
  File "train.py", line 106, in train
    nb_val_samples = 64
  File "/usr/local/lib/python2.7/dist-packages/Keras-2.0.2-py2.7.egg/keras/legacy/interfaces.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/Keras-2.0.2-py2.7.egg/keras/engine/training.py", line 1913, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/usr/local/lib/python2.7/dist-packages/Keras-2.0.2-py2.7.egg/keras/callbacks.py", line 75, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/usr/local/lib/python2.7/dist-packages/Keras-2.0.2-py2.7.egg/keras/callbacks.py", line 477, in on_epoch_end
    if self.monitor_op(current - self.min_delta, self.best):
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

Any ideas why the monitor_op might be NoneType and not a value that can be used to correctly handle the callbacks?

Perhaps it is related to the custom sparse_accuracy_ignoring_last_label and the corresponding loss function... I started debugging but it looks like the monitor_op is correctly assigned.

Okay, I spoke with the author of https://github.com/warmspringwinds/tf-image-segmentation, and he mentioned another key aspect to maximizing performance when training resnet with pretrained weights for segmentation tasks is to lock the batch normalizations so they never change while training segmentation. I'm curious if there is an easy way to do that with Keras.

Thank you for your code!
The label of VOC dataset is .mat file, but when I train my own dataset. My trained images are jpegs, and their labels are pngs.
I meet the following error
Value error: could not broadcast input array from shape(320,320,4) into shape(320,320,1).

I want to know, the dataset must be .mat format or not? Is the label format must be the same as the VOC dataset labels?
Thank you very much!

why did you use the bilinearupsanpling2D you wrote instead of the upsample2D layers from master keras?

Hello,

I have successfully trained the AtrousFCN_Resnet50_16s model with my own data and am having difficulty getting the inference.py module to run. I am hoping that you might have some insight for how I can get the model up and running. Can you help? Thanks! See below:

First a little about the model: My own training data has 3 values: 0 corresponds to the VOID layer in the VOC Data and is the outline of my objects. 1 corresponds to the segmentation pixels I am looking for. And 2 corresponds to background. When I trained the model, I set the number of classes to 2 with the desire/expectation that the model would train to identify categories 1 and 2 in the same fashion as the VOC2011 data. I used the loss function "softmax_sparse_crossentropy_ignoring_last_label" that was included with your code. Here are my results after 250 epochs.

From the shapes of the plots, I believe my model is training correctly. As an output, in the AtrousFCN_Resnet50_16s folder, I get a new file with the timestamp for model completion titled 'checkpoint_weights.hdf5' which I understand as the new weights/biases for the trained model.

Now, I am working on now evaluating the model running evaluation.py using the AtrousFCN_Resnet50_16s model and the new checkpoint file. I have set nb_classes to 2, and pointed the code to use the new checkpoint file for weights. When I run the code I receive the following error:

ValueError: Dimension 0 in both shapes must be equal, but are 1 and 2 for 'Assign_318' (op: 'Assign') with input shapes: [1,1,2048,21], [2,2048,1,1].

This error traces back to line 32 of inference.py: model.load_weights(checkpoint_path, by_name=True)

Interestingly, when I instead point to the weight file I used when I initialized training ('fcn_resnet50_weights_tf_dim_ordering_tf_kernels.h5), I do not get this error. I think the problem is with inference.py interpreting the new model weights file or with the model itself.

Will you verify that I am pointing to the correct model weights file for evaluation?

I also see that line 31 in inference.py in my code reads: `model = globals()[model_name](batch_shape=batch_shape, input_shape=(512, 512, 3))', and line 21 of my code reads: 'batch_shape = (1, ) + image_size + (3,)'.

When I look at my error again (> ValueError: Dimension 0 in both shapes must be equal, but are 1 and 2 for 'Assign_318' (op: 'Assign') with input shapes: [1,1,2048,21], [2,2048,1,1].) I notice that the matrices look reversed, and that the model is looking for 21 classes instead of 2 (last digit versus first digit). What do I need to adjust in my code to fix this error? Do I need to change the batch shape? Is there a spot during model initialization where I am incorrectly loading weights? Do I need to adjust something in the SegDataGenerator to get it to work? Thank you for your input.