Loss functions about midas HOT 5 CLOSED

@ranftlr When I attempted to do a zero-shot test on my own dataset, I found that the results produced by MidasNet are sometimes over-smooth. As far as I am concerned, while the trimmed loss can exclude some points with imperfect groundtruth depth, it may lead to over-smooth boundaries in predicted depth maps, since the depth near the semantic boundaries in images is more difficult to learn. In other words, the points in semantic boundaries may be excluded by the trimming. So I tried finetuning MidasNet with BerHu Loss. The conclusion is consistent with your opinion. While the depth in the semantic boundaries are much sharper, it can also produce some outrageous bad cases.

from midas.

No, we tried neither normal nor BerHu losses. Other than the losses reported in the paper, we experimented with a Cauchy loss for a bit. We found it tricky to set the hyper-parameters correctly, however, and weren't able to achieve better results than the trimming.

from midas.

@ranftlr Thanks for your quick response! Would you please share the insights why choose l1 rather than berhu, since there are some other works that concludes normal loss and berhu loss would effectively improve the performance. Thanks a lot.

from midas.

Conceptually, BerHu does to opposite of what we try to achieve with the trimming. It behaves like L1 for small errors, but amplifies large errors. According to our observations, these errors are mostly due to imperfections in the ground truth and thus do not provide a relevant signal for learning. I'm also not aware of the work that shows a significant advantage of BerHu over L1. Can you point me to it?

As for the normal loss: the gradient matching term is implicitly constraining the normals and very related to standard normal losses. A common formulation for a normal loss would be to measure cosine distance of the normals, which themselves are a function of the gradient (see equation 5 in this paper for example). The resulting loss is a function of the gradient of the estimate and the gradient of the ground truth. So is the gradient matching term. The difference is that the gradient matching term also considers the magnitude of the gradient, an not only the direction. We didn't try the normal loss, so it might or might not improve results. However, even if it does, I don't believe that the difference would be significant.

from midas.

Thanks for sharing this information. Yes, we do observe issues with over smooth results in some cases. The gradient matching term does counter it to some degree, but also cannot do it when the ground truth is already missing values.

from midas.