Nice! Would you release your implementation of the loss function? about midas HOT 14 CLOSED

Hi! Please see: https://gist.github.com/dvdhfnr/732c26b61a0e63a0abc8a5d769dbebd0

Hi this link is failed, could you share a new one
Thanks.

from midas.

Hi!
Please see: https://gist.github.com/dvdhfnr/732c26b61a0e63a0abc8a5d769dbebd0

from midas.

@dvdhfnr I implemented the loss by myself. I found that the range of prediction is unstable. It can be -3 -- 8, -10 -- 30, -200 -- 1000, in differernt training attempts. I wonder if your met the problem when you trained the network. Thank you very much.

from midas.

No, I did not met this problem. Did you scale/shift the target depths to [0, 10], as described in the paper?

from midas.

@dvdhfnr I train my model on NYUDepthV2, and scale/shift the depth value by

mask  = target > 0
target[mask] = (target[mask] - target[mask].min()) / (target[mask].max() - target[mask].min()) * 9 + 1
target[mask] = 10. / target[mask]
target[~mask] = 0.

How did you scale/shift the depth map?

from midas.

To be more precise:
"For all datasets, we shift and scale the ground-truth inverse depth to the range [0, 10]."

from midas.

I test the provided midas_loss. It also leads to unstable output range. The output range is influenced by the initial weights.
I trained the network with different initial weights, and got the following results, in which the three numbers are the min, mean and max of prediction:

Maybe there should be additional constraints on the prediction to restrict the output range.
Finally, I am looking forward to the scripts that are used to produce the data set.

from midas.

The predictions resemble the target inverse depths up to a scale and shift.

from midas.

As @dvdhfnr mentioned: This is by construction, as the value of the loss is independent of the output scale. Thus the ranges are dependent on the initialization. If you need a fixed scale range, you could try to add a (scaled) sigmoid to the output activations. However, this might change training behavior and you likely need a sensible initialization.

from midas.

To be more precise: "For all datasets, we shift and scale the ground-truth inverse depth to the range [0, 10]."

@dvdhfnr Hi, I found the normalization range is [0, 1] in original paper, so which range is correct?

thx

from midas.

For the latest version of the paper we used the range [0, 1] (https://arxiv.org/pdf/1907.01341v3.pdf, Page 7).

#2 (comment) was referring to an earlier version (https://arxiv.org/pdf/1907.01341v1.pdf, Page 6).

from midas.

@dvdhfnr Hi, did you use this loss function for MiDaS v3.0 as well?

from midas.

@dvdhfnr @ranftlr Hi,
I want to know how did you rescale the ground truths depth to range [0,1].
As my understanding, the following steps are done:

1. Get the ground truth inverse depth
2. Normalize like this inverse_depth_normalization = (inverse_depth- inverse_depth.min()) / (inverse_depth.max() - inverse_depth.min())

Does my understanding right? if not, could you please tell us how do you do "For all datasets, we shift and scale the ground-truth inverse depth to the range [0, 1]."?

Thx.

from midas.

@Twilight89
Same problem. I think the author will never reply. Do you have any conclusion? Or do you implement the training process?

from midas.