Gradients can not be obtained in SDF. about torch-ngp HOT 5 CLOSED

Thanks for your great implemention! However, it seems that when the gradient of SDF versus xyz is added in loss terms, the backward pass fails with following error:

Traceback (most recent call last):
  File "train.py", line 178, in <module>
    main()
  File "train.py", line 148, in main
    train_loss.backward()
  File "/home/xxx/anaconda3/envs/torch/lib/python3.8/site-packages/torch/_tensor.py", line 255, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/xxx/anaconda3/envs/torch/lib/python3.8/site-packages/torch/autograd/__init__.py", line 147, in backward
    Variable._execution_engine.run_backward(
RuntimeError: trying to differentiate twice a function that was markedwith @once_differentiable

The loss function looks like:

gradient, = torch.autograd.grad(sdf, [xyz], grad_outputs=grad_outputs, create_graph=True)
loss = torch.abs(gradient.norm(dim=-1) - 1)

from torch-ngp.

@MingwuZheng Which backbone are you using? For the implementation in this repo, it has been fixed in 197fd4d. For TCNN, you may check their implementation.
However, I haven't tested this type of training, so it may lead to unstable/buggy results. Feel free to report if you meet other problems ;)

from torch-ngp.

@MingwuZheng Which backbone are you using? For the implementation in this repo, it has been fixed in 197fd4d. For TCNN, you may check their implementation. However, I haven't tested this type of training, so it may lead to unstable/buggy results. Feel free to report if you meet other problems ;)

@ashawkey Thanks for your reply! I am using simple nn.linear instead of TCNN or FFMLP, and I did fix this issue following 197fd4d. Despite that, the reconstructed results are not desirable. Specifically, the sdf value errors on sampled points are low but the gradients (normal vectors) are noisy and not correct. This is partly due to my surface sampling strategy (without pertubing), but I am wondering perhaps it is mainly because a SMOOTH INTERPOLATION (see Appendix A) is not implemented in your code, making the gradients not smooth under a less constrained sampling strategy. Could you confirm this for me? : )

from torch-ngp.

@MingwuZheng Currently I don't have a use case, but I'll definitely try later. Besides, TCNN's pytorch extension supports smooth step interpolation for hash encoding, and maybe you can try with that first?

from torch-ngp.

@ashawkey Thank you, I will try TCNN first, and report if I found anything interesting. Besides, you may refer to SIREN implementation for more information on surface sampling. If you have any question or would like to discuss neural implicit representation, please feel free to contact me through email. : )

from torch-ngp.