Homework 3 of Deep Learning class on Electrical Engineering Master's at PUC-Rio

This Project has the objective of learning how a semantic segmentation network works, implementing more specifically U-Net, in a remote sensing dataset. As long as the dataset has a huge imabalance problem we used weighted cross entropy loss, which is also implemented in tensorflow 2.x (keras), to handle class imbalance problem. The idea of the project is to compare the results among different trails of the same dataset extracted with different patch sizes, since remote sensing images have a quite big resolution and see how the patch size affects the performance of the network.

The number of pixels by class can be seen on the image below, to show clearly the imabalance problem:

The weights calculation of each class for the loss function can be seen below:

If this weight calculation doesn't fit for you, you can try a normalized version (the sum of all classes are equal to 1).

inside the notebook train_model.ipynb you have all the necessary scripts to train U-Net, including U-Net architecture implementation, patches extraction, weighted cross entropy custom loss and more.

I used a stride with half the size of patch size, to enhance the the size of training data, since I had only one image to train. Also, if you have a powerful GPU, use a bigger patch size.

The model hyperparameters of the three trials, can be seen below:

If you want to reproduce results the dataset is in this drive: https://drive.google.com/drive/folders/17sy0-ljxP9xQH8ANvjsZTru8aCOxZz4x?usp=sharing

On testing notebook Test_model.ipynb you have all the preprocess scripts and also prediction patches reconstruction and patches visualization code.

We can see an intresting observation in model training histories. As long as we reduce the patch size more noisy is the training:

Also, looking into the metrics we can see model 1, with the greater patch size, is the best model.

Test reference

Model 1 prediction reconstruction

Model 2 prediction reconstruction

Model 3 prediction reconstruction