nianticlabs / wavelet-monodepth Goto Github PK

Hi,

Thanks for the brilliant work! I notice the inverse DWT you use in the code is IDWT(). However, I didn't find its usage in the current documentation of the Pytorch Wavelets library. I wonder which version of Pytorch Wavelets did you use in the paper? And if I want to use the DWT transformation under your code settings, am I supposed to use a function like DWT(img)?

Thanks a lot!

Hi,
I have well understood the flow then, I can see that :
"self.is_test" always remain False, hence the control will never enter in "If" condition

Could you please tell me why you are doing either multiply by 1000 or dividing by 1000.

Another point, as I am working with the smaller version of the NTU dataset i.e. "nyu_depth_v2_labeled.mat", whether this step of divide or multiply is also applicable in this dataset also ?

(wavelet-mdp) enb@enb-MS-7D42:/media/enb/d246805e-89c3-40bf-95da-9f0d81ea7b05/home/enb/ZYF/wavelet-monodepth-main$ python KITTI/test_simple.py --image_path /media/enb/d246805e-89c3-40bf-95da-9f0d81ea7b05/home/enb/ZYF/Video_test/photo2/set1/ --model_path pretrain/ours_mobilenet_nyuv2.pth --encoder_type mobilenet
-> Loading model from pretrain/ours_mobilenet_nyuv2.pth
Building network
Encoder... Building mobilenet... Traceback (most recent call last):
File "KITTI/test_simple.py", line 185, in
test_simple(args)
File "KITTI/test_simple.py", line 81, in test_simple
encoder = make_depth_encoder(args)
File "/media/enb/d246805e-89c3-40bf-95da-9f0d81ea7b05/home/enb/ZYF/wavelet-monodepth-main/KITTI/networks/network_constructors.py", line 14, in make_depth_encoder
use_pretrained = opts.weights_init == "pretrained"
AttributeError: 'Namespace' object has no attribute 'weights_init'

Hello,

Thanks for the impressive work! I've cloned the code and setting the environment as required (pytorch 1.7.1, torchvision 0.8.2). When running the code on KITTI (w/o depth hints), I encountered two problems.

1. There is a warning 'UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule.' Consider I use the 1.7.1 version torch, do I need to change the order in run_epoch.py as suggested by the warning?

2. An error shows in line 231 of mono_dataset.py: inputs[("depth_gt", scale)] = self.resizescale
   'TypeError: img should be PIL Image. Got <class 'numpy.ndarray'>'
   I transform 'depth_gt' to PIL Image format and the problem is settled. I wonder if the error is an individual case for me, or how do you handle this problem.

3. There is another small error when using grid_sample(): 'UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.' Do I need to change the 'align_corners' parameter or just leave it unchanged?

I wonder how do you settle these issues in your implementation. Thank you a lot!

Hi,

When I run the code on KITTI, I can not get the scores reported on the paper. I run the code with stereo training, with 1024x320 resolution and wavelet decomposition, the command is shown below:
"train.py --data_path --log_dir --encoder_type resnet --num_layers 50 --width 1024 --height 320 --frame_ids 0 --use_stereo --split eigen_full --num_epochs 300 --use_depth_hints --depth_hint_path --use_wavelets"

I evaluate the model on Epoch 20 and got the result:
" abs_rel | sq_rel | rmse | rmse_log | a1 | a2 | a3 |
& 0.1146 & 0.8996 & 4.8552 & 0.2024 & 0.8582 & 0.9519 & 0.9785 \",
which is different from the reported socres in the paper :
"0.097 & 0.718 & 4.387 & 0.184 & 0.891 & 0.962 & 0.982"

I use pytorch 1.7.1 (with cuda 10.1) , torchvision 0.8.2 (with cuda 10.1), pytorch-wavelets 1.3.0, numpy 1.19.5, opencv 3.4.2, pillow 6.2.1, scikit-learn 0.24.2, on python 3.7.10. The setting is consistent with that suggested by the github repository.

So I wonder how to produce to desired scores, is there anything wrong with my settings? Thank you for your advice :D

Hello, what is the function of 2 **2 when obtaining 'h'

h = (2 ** 2) * self.wave1(x_d1).unsqueeze(1)

and other examples, including
ll = (2 ** 3) * self.wave1_ll(x_d1)
h = (2 ** 1) * self.wave2(x_d2).unsqueeze(1)

Could you please tell the function of that? Thank you in advance.

Hi,

Here comes a new problem.
When I trained the model with the resolution of 192*640 with command
train.py --data_path <path> --log_dir <path> --encoder_type resnet --num_layers 50 --width 640 --height 192 --frame_ids 0 --use_stereo --split eigen_full --num_epochs 30 --use_depth_hints --depth_hint_path <path> --use_wavelets
, the training process was stuck. The logging message of the first epoch did not show. There was no other message nor error reported. It's pretty weird because the resolution is not supposed to interfere with the training process. Can you please help to debug that? Thank you!

Hello, when I use the training command without depth hints ,the loss is very small , I'm not sure whether this is the normal phenomenon.
The loss is displayed as 0. 0000