ternaus / kaggle_dstl_submission Goto Github PK

Dear all,

I'm trying to use the NN with satellite pictures with real coordinates and different projection, is there anything I should change or adapt? Right now I'm getting a very big loss:
8832/25600 [=========>....................] - ETA: 15210s - loss: 39.2488 - binary_crossentropy: 0.2799 - jaccard_coef_int: 0.5290

Does it make any difference if the images are not on the 0,0 with the top left corner?

Bests
G.

Thanks so much for your code, I have learned a lot from it.
I have a question just bugging me,
how did you choose the number of epochs for each unet? was it just through a train_test validation split or through cross validation?
Also, generally, how do you choose the number of epochs for large computer vision data sets and do you always perform cross validation?

Thanks again.

After running visualize I get the following error: [Errno 2] No such file or directory: 'predictions/Objects--6080_4_4.png'.
I've followed all the steps. Is there something wrong I didn't pay attention to?
Thank you so much.

When I run your code -cache_train.py in spyder,it told that --TypeError: 'compression_opts' is an invalid keyword argument for this function(I have installed all packages what we need ),Can you tell me what should I do?
Thanks

How can I label my own images so that this source code would work on them?
Thank you

I have a problem statement in which there are total 16 classes. I trained with Segnet with all classes at once. Results are not that good.
And I saw your solution.
I wanted to know which one is more suitable:

1. training for each of the classes separately
2. training for all classes at once.

Did you tried training of all classes at once. I just wanted to know you experience and suggestions regarding this.

I ran the make_prediction_cropped_buildings.py ,after below classes

1. python get_3_band_shapes.py
2. cache_train.py

got the ValueError.

python make_prediction_cropped_buildings.py
Using TensorFlow backend.
Traceback (most recent call last):
File "make_prediction_cropped_buildings.py", line 26, in
model = read_model()
File "make_prediction_cropped_buildings.py", line 21, in read_model
model = model_from_json(open(os.path.join('../src/cache', json_name)).read())
File "/home/darshan/anaconda2/lib/python2.7/site-packages/keras/models.py", line 214, in model_from_json
return layer_from_config(config, custom_objects=custom_objects)
File "/home/darshan/anaconda2/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 44, in layer_from_config
custom_objects=custom_objects)
File "/home/darshan/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 2596, in from_config
process_layer(layer_data)
File "/home/darshan/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 2592, in process_layer
layer(input_tensors)
File "/home/darshan/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 1469, in call
node_indices, tensor_indices)
File "/home/darshan/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 1379, in _arguments_validation
'Layer shapes: %s' % (input_shapes))
ValueError: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer shapes: [(None, 1024, 14, 7), (None, 256, 14, 14)]

Can any one help me on this?

Thanks

Hi!
Sorry for raising an issue, I found it as a way of reaching out to you.
I am very impressed by your solutions for many image segmentation competitions, where participants are provided with image and ground-truth labels.
However I wanted to ask about your approach in cases when ground truth is only point, not mask (for example https://www.datasciencechallenge.org/challenges/1/safe-passage competition).
Could you reveal your appropach to such a problem? How to prepare masks in such a case, what kind of networks to use?

Kind regards,
Tomasz

Dear all,
I see that in the code the img_rows and img_cols is set to 112, has that a special meaning or is it the best value that you found during the training?

Thanks
G.

Dear all,

I was able to run the code with the Kaggle image set, I would like to perform some predictions on sattelite photos of mine, but they are only in RGB (I don't have any Panachormatic and others wavelength images), is this possible?

Thanks
G.

When I try to run unet_buildings.py, I encounter this issue. I changed the channel from 16+3 to only RGB channels but I don't think that will cause the issue.

Here are the traceback:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/root/anaconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/root/anaconda3/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 429, in data_generator_task
generator_output = next(self._generator)
TypeError: 'threadsafe_iter' object is not an iterator
Epoch 1/50

Traceback (most recent call last):
File "unet_buildings.py", line 278, in
nb_worker=8
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1532, in fit_generator
str(generator_output))
ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None

Dear all,

when I try to perform a prediction on the buildings (after the training) I get the following issue:

Using TensorFlow backend.
2017-12-04 14:24:44.615883: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2
0%| | 0/425 [00:00<?, ?it/s]Traceback (most recent call last):
File "/home/apache/kaggle_dstl_submission/src/make_prediction_cropped_buildings.py", line 95, in
result += [(image_id, mask_channel + 1, mask2poly(new_mask, threashold, x_scaler, y_scaler))]
File "/home/apache/kaggle_dstl_submission/src/extra_functions.py", line 125, in mask2polygons_layer
contours, hierarchy = cv2.findContours(((mask == 1) * 255).astype(np.uint8), cv2.RETR_CCOMP, cv2.CHAIN_APPROX_TC89_KCOS)
ValueError: too many values to unpack

Can you please help?

Bests
G.

when i use jaccard as loss function, the training loss dose not go down but go up. what is wrong with me?

Hi, I would like to know under which license is your work available. GitHub's default license is "all rights reserved" for public repositories, such as this one.

I have read the following, about winners' obligations of publishing the solution: https://www.kaggle.com/c/dstl-satellite-imagery-feature-detection/rules
I am not exactly sure of what type of competition this is, so I am not sure under which license you have published your solution according to the rules. I would like to know it. Adding a license to your repository is simple:
https://help.github.com/articles/adding-a-license-to-a-repository/
Just be sure that in the template you pick to add your names.

By the way, the following piece of code is licensed Apache 2.0: https://www.kaggle.com/ceperaang/lb-0-42-ultimate-full-solution-run-on-your-hw/code/code
And it also use other Apache 2.0 Licensed content. But the license for that kernel may not be the same as for this GitHub repository.

Many thanks!

How can I make this code work with images of 3 bands only?

Hi.
Great work and thanks for code.
I was trying to repeat your results on building class (which look impressive) and I just ran your make preduction building file with the model saved in the cache without making any changes in the code. I evaluated the output csv and noticed that the reported 70%ish performabce drops to roughly 35%. Have you changed the building prediction file that the results do not match?

Thanks.

There is no compression and compression_opts arguments for h5py.File. It only exists in create_dataset? Is it a bug and how to fix it?

I want to implement your pre-trained model of DSTL competition only on RGB images. I don't have data set of other band images for my satellite imagery application. Is it possible to test only RGB images using your pre-trained model?

Hi,
In your code, the unet model in 'unet_structure.py', 'unet_building.py', 'unet_road.py' seem to be the same? I have not found any difference in the model definition. What is the point of using different model for different objects?
Best regards!

I run you code "unet_buildings.py" to train model. then I got an error as following:

[2017-11-16 09:49:53.996358] Creating and compiling model...
Traceback (most recent call last):
File "unet_buildings.py", line 254, in
model = get_unet0()
File "unet_buildings.py", line 104, in get_unet0
up6 = merge([UpSampling2D(size=(2, 2))(conv5), conv4], mode='concat', concat_axis=1)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 1680, in merge
name=name)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 1299, in init
node_indices, tensor_indices)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 1371, in _arguments_validation
'Layer shapes: %s' % (input_shapes))
ValueError: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer shapes: [(None, 2, 14, 512), (None, 2, 14, 256)]

I use keras+Theano+GPU.
Any idea why it is crashing?

Many Thanks!

When I run the file cache_train.py, it says that:

File "C:\Users\144601\AppData\Local\Continuum\anaconda3\envs\py36\lib\site-packages\h5py_hl\files.py", line 124, in make_fapl
raise TypeError(msg)
TypeError: 'compression' is an invalid keyword argument for this function