Thanks so much about kaggle_dstl_submission HOT 1 CLOSED

Thank you.

[1] It is pretty hard to overfit with Unet, although it is still possible. Typically both train and validation loss get to the platoe roughly at the same time. So one may choose when to stop or lower learning rate based on the train loss and not validation loss as it should be done with other types of the problems or network architectures.

=> 50 epochs worked really well, so for most of the classes we trained 50 epochs with lr = 1e-3, and another 50 with lr=1e-4

Most likely, we could use 40 or 60 or 100 and the result would be the same. As long as you get to the platoe it does not really matter.

[2] In this particular dataset train, public and private test were so different and so small that none of the validation techniques that we tried worked. And we tried to use several full images for validation, we tried to divide each image into 80% train and 20% validation and crop from there. No success. So in this particular problem, we validated with our eyes (dumping all test predictions with corresponding images and looked at them) + we used feedback from leaderboard, it was not really reliable but was able to catch big imporvements.

[3] I always do validation, when dataset is big and balanced I simply take hold out set, but typically it is sometimes more complicated because usually data is dirty and full of outliers.

[4] My standard approach to different all neural networks related problems is to use validation loss for early stopping, adjusting learning rate schedule (increase/decrease learning rate).

from kaggle_dstl_submission.