Help training out bright reflection... about manydepth HOT 1 OPEN

Hi - and thanks for your interest in the paper!

Great to see ManyDepth (kind of) working on another dataset!

Yes, incorrect predictions on reflections like you see are an unfortunate result of training with a self-supervised loss - where the minimum SSIM loss will correspond to an incorrect depth value.

The cost volume in ManyDepth allows the network to make sensible predictions for the upper left since it most likely can get good matches in this region - Monodepth struggles due to ambiguity.

However it is also the cost volume which will hurt predictions for reflections for ManyDepth - the best match in the cost volume will be wrong, since it assumes a static, lambertian world, and the network just trusts the cost volume here. This is very similar to what we observe in the paper with moving objects - and is why we use a teacher network (i.e. MonoDepth2) to constrain the predictions.

Hope this helps!

from manydepth.