dvlab-research / ref-npr Goto Github PK

hi~thanks for your excellent work🤣
I also use controlnet to generate the truck in snow, but unfortunately, I can't use refnpr to get satisfactory results.
I want to know if the problem is that controlnet generates images that are integer multiples of 64 in length and width, and then resizing them to the same size as the original image results in this error.
I would also like to ask you how to use controlnet to get an image of the same size as the reference image.

Hi, I wonder how did you compute your robustness metric and Ref-LPIPS as mentioned in Figure B.1.
I noticed you compute PSNR, SSIM and LPIPS in calc_metric.py but that seems to be computed on stylized view and original view images. Did you provide codes that work for what you mentioned in Figure B.1. and Table 1? Thanks very much.

Hi,thanks your amazing work.There is a problem.
No such file or directory:'./exps/base_pr/ckpt_svox2/llff/flower/ckpt.npz
Can you share your ckpt.npz?Thanks a lot.

I attempted the third step to render, but encountered the following error. Could you share your ckpt file? Thanks!

FileNotFoundError: [Errno 2] No such file or directory: './exps/refnpr/flower_colorful/exp_out/ckpt.npz/ckpt.npz'

When I was training the ref-npr model, I met such a traceback:

Traceback (most recent call last):
File "./exps/refnpr/ref_regist.py", line 145, in
xyz_min = torch.minimum(xyz_min, torch.min(xyz_pos[depth_rep!=0].reshape(-1, 3), dim=0)[0])
IndexError: min(): Expected reduction dim 0 to have non-zero size.

It seems that for some dataset, during the iteration, depth_rep can be all 0 and xyz_pos[depth_rep] can therefore be torch.tensor([]).
How can I sovle the issue, can I firstly check torch.any(depth_rep) to deside whether running the following codes? Like:

if torch.any(depth_rep):
        xyz_min = torch.minimum(xyz_min, torch.min(xyz_pos[depth_rep!=0].reshape(-1, 3), dim=0)[0])
        xyz_max = torch.maximum(xyz_max, torch.max(xyz_pos[depth_rep!=0].reshape(-1, 3), dim=0)[0])
else:
        pass

Besides, xyz_min and xyz_max seems to stay stationary before the error occurs.

Great work! I currently have trouble with the code. When I run the code (at step 2 and step 3), there is always a traceback:
Traceback (most recent call last):
File "opt.py", line 471, in
eval_step()
File "opt.py", line 406, in eval_step
rgb_pred_test = grid.volume_render_image(cam, use_kernel=True)
File "....../anaconda3/envs/RefNPR/lib/python3.8/site-packages/svox2/svox2.py", line 1210, in volume_render_image
and imrend_fn_name in _C.dict
AttributeError: 'NoneType' object has no attribute 'dict'
How can I solve the issue, is there a problem when creating the environment? Thank you so much!

Great work. I appreciate that you share the codes.

How do these losses involve in the backwards?. In this part, just compute the output from the model.
do they internally calculate loss?

https://github.com/dvlab-research/Ref-NPR/blob/main/exps/refnpr/ref_opt.py#L326-L354

Congratulations on Your excellent work, but I find the result on tnt dataset is bad. The image cannot reconstruct well in the first stage. Could you please check the result.

Hello, thank you for your excellent work. I'd be happy to help you with the issues you're encountering.

When I run the command bash ../exps/base_pr/run_all_bases_llff.sh, I encounter the following error:
AttributeError: 'NoneType' object has no attribute 'dict'
How can I resolve this?

When I skip the second step and proceed to the third step, it says I'm missing the ckpt file.
FileNotFoundError: [Errno 2] No such file or directory: './exps/refnpr/flower_colorful/exp_out/ckpt.npz/ckpt.npz'
How can I obtain the ckpt file for testing purposes?

Hi author,

Thanks for posting the work. I have quick question about ray registration. From Line 227, it seems that the dictionary is a tensor that can only store a single ray. In case of multiple rays falling into a single grid, the grid will be overwritten by the latest ray.
Is that correct?

I have successfully run Step 1 and 2. But when I run Step 3:bash ./exps/refnpr/run_single.sh ./data/ref_case/flower_colorful/, I meet the error that: "FileNotFoundError: [Errno 2] No such file or directory: './exps/refnpr/flower_colorful/exp_out/ckpt.npz/ckpt.npz'.

I found that in Step 2, I have got the file "./exps/base_pr/ckpt_svox2/llff/flower/ckpt.npz"
BUT in Sep 3, I need the file './exps/refnpr/flower_colorful/exp_out/ckpt.npz/ckpt.npz', which is not generated before.

hello, when I run the step 2,
meet the error Segmentation fault

and here is the svox2 implement

and the torch enviroment

Importing svox2 package under RefNPR environment will report the following error. But I can import svox2 package under base environment.