baegwangbin / surface_normal_uncertainty Goto Github PK

I attempted to test the NYU dataset using the provided weight file and obtained a result that was consistent with 85.17 in the paper.

But when I tried to train the NYU dataset, it was difficult to achieve the same accuracy and the gap was significant.

I have not made any changes to the training configuration of train, using a 3090TI.

(test) jiehu@jiehu-Z490-AORUS-MASTER:/media/jiehu/hard_disk/work_data_hd/code/open_source/surface_normal_uncertainty$ python test.py --pretrained scannet --architecture BN
loading checkpoint... /media/jiehu/hard_disk/work_data_hd/code/open_source/surface_normal_uncertainty/checkpoints/scannet.pt
Using cache found in /home/jiehu/.cache/torch/hub/rwightman_gen-efficientnet-pytorch_master
Downloading: "https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/tf_efficientnet_b5_ap-9e82fae8.pth" to /home/jiehu/.cache/torch/hub/checkpoints/tf_efficientnet_b5_ap-9e82fae8.pth
Loading base model ()...ffff: <_io.BufferedReader name='/home/jiehu/.cache/torch/hub/checkpoints/tf_efficientnet_b5_ap-9e82fae8.pth'>
Done.
Removing last two layers (global_pool & classifier).
fpath: /media/jiehu/hard_disk/work_data_hd/code/open_source/surface_normal_uncertainty/checkpoints/scannet.pt
ffff: <_io.BufferedReader name='/media/jiehu/hard_disk/work_data_hd/code/open_source/surface_normal_uncertainty/checkpoints/scannet.pt'>
Traceback (most recent call last):
File "test.py", line 97, in
model = utils.load_checkpoint(checkpoint, model)
File "/media/jiehu/hard_disk/work_data_hd/code/open_source/surface_normal_uncertainty/utils/utils.py", line 49, in load_checkpoint
ckpt = torch.load(fpath, map_location='cpu')['model']
File "/home/jiehu/anaconda3/envs/test/lib/python3.8/site-packages/torch/serialization.py", line 713, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/jiehu/anaconda3/envs/test/lib/python3.8/site-packages/torch/serialization.py", line 920, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '<'.

maybe the checkpoint file is not correct, please help check it, thanks.

Thank you for sharing your code!

I'm quite new to the normal estimation area and wonder what is the best pre-trained model for cars and what is the coordinate that defines normal.

Which one is the correct one?

Thank you.

Thanks for your great work!I want to know if the uncertainty map is the kappa of the image?Why it needs to multiple -1?
If the beta*N is the points selected by uncertainty guided to compute loss and what does the coverage mean?

Hi Bae:
Thank you for your great work here! I am trying to compare with your work on ScanNet v2 but do not know the exact details. I wonder if it is available to provide: Image size and validation split here?

I am now using a size of 480(h)*640(w) and a validation split up to 60000 samples (from framenet)! It takes a long time to validate once. I am not sure whether this is correct.

Thank you for your excellent work！Because there is no code for data processing, could you please introduce how to align the normal parameters output by the network with the GT in the FrameNet dataset (mentioned in the paper).

Hi, thanks so much for your excellent work on the surface normal estimation. When I was training on the taskonomy dataset with the paper-proposed loss function 'UG_NLL_ours' (i.e., simply replace the img and norm paths of the Nyuloader), the loss tended to decrease and even be negative. Is it okay for training? Or there may exist some mistakes.

Thanks for your great work!But when I trained the model on my custom dataset with --use_baseline&&--loss_fn NLL_ours,and load run with python test.py,it shows:

How can I solve this?
Thank you.

Hi,

This is a great work! Would you consider adding your model to kornia? https://github.com/kornia/kornia
I believe, that it could boost the usage and citations for you and make life a bit simpler for your users.

--
Best, Dmytro

Hi Gwangbin,

Thanks for sharing your great work!
I am curious about the training process, is the network trained by end-to-end training?

Best wishes,
Runsong

Hello,

When I run python test.py --pretrained scannet --architecture BN on Win10, it shows this:

loading checkpoint... ./checkpoints/scannet.pt
Loading base model ()...Using cache found in C:\Users\LZS/.cache\torch\hub\rwightman_gen-efficientnet-pytorch_master
Done.
Removing last two layers (global_pool & classifier).
Traceback (most recent call last):
File "test.py", line 97, in
model = utils.load_checkpoint(checkpoint, model)
File "F:\surface_normal_uncertainty\utils\utils.py", line 57, in load_checkpoint
ckpt = torch.load(fpath, map_location=lambda storage, loc: storage)['model']
File "F:\Anaconda3\lib\site-packages\torch\serialization.py", line 608, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "F:\Anaconda3\lib\site-packages\torch\serialization.py", line 777, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '<'.

I have tried directly downloading the model file, tf_efficientnet_b5_ap-9e82fae8.pth, from https://zzun.app/repo/rwightman-pytorch-image-models-python-deep-learning#releases, and use it to replace the original one.

I checked their file size and they are all over 100M.

Could you please give me some idea?

Hello, I didn't find the data loader for Scannet, could I modify the file dataloader_nyu.py to load scannet data? Are there any specific modifications required for scannet? Thanks so much.

I wanted to try to train a new model using my own dataset, and when using NLL_ours as the Loss function, the loss value would gradually become negative during training. While training is normal when using L2 or AL, I don't know how to solve it. Looking forward to your reply.

Hi,
It is really an awesome job. I am looking forward to the training part.