sicxu / deep3dfacerecon_pytorch Goto Github PK

I notice that in compute_shape(), you only use mean shape, but not mean expression. I understand that it is because you use an expression basis provided by Guo et al, instead of those in 01_MorphableModel.mat.

However, in Exp_Pca.bin, there is a variable called expMU which can be converted to the mean expression. Why don't you use it?

All res is with img size 224. If I use original image size mesh.
How can I inverse the transform mat

Can you tell me the configuration of the project, such as the version of CUDA, cudnn, python, and platform Linux/Windows?
Thanks!

Thank you for your amazing job! Two question: (1) If I want to modify size of input image(eg. 224 -> 112), How can I set focal, z_near and z_far? (2) why choose focal == 1015 for image size 224? Looking for your response~

Thanks for your work, it's very helpful for 3d face recognition.
The question is that how can i use the Arcface work to recognize a 3d face. There's no related content in the README file.

Thank you for your excellent job!
In your work, the resolution of all input images will be processed as 224 * 224. Now I want to change the resolution to 512 * 512, and then I find that if I want to change the resolution, changing the five rendering parameters(focal, center, camera_d, z_near, z_far) is necessary. But I don't know how to understand the parameter "camera_d" now.

Thanks for the work, it is very helpful. Will it be possible to share the evaluation code to get the Reconstruction accuracy?

Hi, my issue is similar to #2 , but not exactly the same.

Traceback (most recent call last):
File "/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1515, in _run_ninja_build
env=env)
File "/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "test.py", line 72, in
main(0, opt,opt.img_folder)
File "test.py", line 62, in main
model.test() # run inference
File "/media/a706/新加卷/xyj/workspace/Deep3DFaceRecon_pytorch/models/base_model.py", line 162, in test
self.forward()
File "/media/a706/新加卷/xyj/workspace/Deep3DFaceRecon_pytorch/models/facerecon_model.py", line 139, in forward
self.pred_vertex, self.facemodel.face_buf, feat=self.pred_color)
File "/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in call_impl
result = self.forward(*input, **kwargs)
File "/media/a706/新加卷/xyj/workspace/Deep3DFaceRecon_pytorch/util/nvdiffrast.py", line 58, in forward
self.glctx = dr.RasterizeGLContext(device=device)
File "/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/ops.py", line 151, in init
self.cpp_wrapper = get_plugin().RasterizeGLStateWrapper(output_db, mode == 'automatic', cuda_device_idx)
File "/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/ops.py", line 84, in get_plugin
torch.utils.cpp_extension.load(name=plugin_name, sources=source_paths, extra_cflags=opts, extra_cuda_cflags=opts, extra_ldflags=ldflags, with_cuda=True, verbose=False)
File "/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 974, in load
keep_intermediates=keep_intermediates)
File "/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1179, in jit_compile
with_cuda=with_cuda)
File "/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1279, in write_ninja_file_and_build_library
error_prefix="Error building extension '{}'".format(name))
File "/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1529, in run_ninja_build
raise RuntimeError(message)
RuntimeError: Error building extension 'nvdiffrast_plugin': [1/14] :/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cu -o rasterize.cuda.o
FAILED: rasterize.cuda.o
:/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cu -o rasterize.cuda.o
/bin/sh: :/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc: No such file or directory
[2/14] :/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/interpolate.cu -o interpolate.cuda.o
FAILED: interpolate.cuda.o
:/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/interpolate.cu -o interpolate.cuda.o
/bin/sh: :/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc: No such file or directory
[3/14] :/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/texture.cu -o texture.cuda.o
FAILED: texture.cuda.o
:/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/texture.cu -o texture.cuda.o
/bin/sh: :/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc: No such file or directory
[4/14] :/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/antialias.cu -o antialias.cuda.o
FAILED: antialias.cuda.o
:/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/antialias.cu -o antialias.cuda.o
/bin/sh: :/usr/local/cuda-10.1:/usr/local/cuda-10.1/bin/nvcc: No such file or directory
[5/14] c++ -MMD -MF common.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/common.cpp -o common.o
FAILED: common.o
c++ -MMD -MF common.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/common.cpp -o common.o
/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/common.cpp:9:10: fatal error: cuda_runtime.h: No such file or directory
#include <cuda_runtime.h>
^~~~~~~~~~~~~~~~
compilation terminated.
[6/14] c++ -MMD -MF texture.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/texture.cpp -o texture.o
FAILED: texture.o
c++ -MMD -MF texture.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/texture.cpp -o texture.o
In file included from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/framework.h:37:0,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/texture.cpp:9:
/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
#include <cuda_runtime_api.h>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
[7/14] c++ -MMD -MF torch_antialias.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_antialias.cpp -o torch_antialias.o
FAILED: torch_antialias.o
c++ -MMD -MF torch_antialias.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_antialias.cpp -o torch_antialias.o
In file included from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/../common/framework.h:37:0,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_common.inl:10,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_antialias.cpp:9:
/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
#include <cuda_runtime_api.h>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
[8/14] c++ -MMD -MF torch_interpolate.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_interpolate.cpp -o torch_interpolate.o
FAILED: torch_interpolate.o
c++ -MMD -MF torch_interpolate.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_interpolate.cpp -o torch_interpolate.o
In file included from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/../common/framework.h:37:0,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_common.inl:10,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_interpolate.cpp:9:
/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
#include <cuda_runtime_api.h>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
[9/14] c++ -MMD -MF rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cpp -o rasterize.o
FAILED: rasterize.o
c++ -MMD -MF rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cpp -o rasterize.o
In file included from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/framework.h:37:0,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.h:41,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cpp:9:
/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
#include <cuda_runtime_api.h>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
[10/14] c++ -MMD -MF torch_bindings.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_bindings.cpp -o torch_bindings.o
FAILED: torch_bindings.o
c++ -MMD -MF torch_bindings.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_bindings.cpp -o torch_bindings.o
In file included from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/../common/framework.h:37:0,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_common.inl:10,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_bindings.cpp:9:
/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
#include <cuda_runtime_api.h>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
[11/14] c++ -MMD -MF torch_texture.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_texture.cpp -o torch_texture.o
FAILED: torch_texture.o
c++ -MMD -MF torch_texture.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_texture.cpp -o torch_texture.o
In file included from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/../common/framework.h:37:0,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_common.inl:10,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_texture.cpp:9:
/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
#include <cuda_runtime_api.h>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
[12/14] c++ -MMD -MF glutil.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.cpp -o glutil.o
FAILED: glutil.o
c++ -MMD -MF glutil.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.cpp -o glutil.o
In file included from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/framework.h:37:0,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.cpp:13:
/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
#include <cuda_runtime_api.h>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
[13/14] c++ -MMD -MF torch_rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_rasterize.cpp -o torch_rasterize.o
FAILED: torch_rasterize.o
c++ -MMD -MF torch_rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem :/usr/local/cuda-10.1:/usr/local/cuda-10.1/include -isystem /home/a706/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_rasterize.cpp -o torch_rasterize.o
In file included from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/../common/framework.h:37:0,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_common.inl:10,
from /home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_rasterize.cpp:9:
/home/a706/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
#include <cuda_runtime_api.h>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.

Hi! I tried to train the network on my own datasets. When I was preparing the landmarks by running the script "python data_preparation.py --img_folder <folder_to_training_images>", I got the following error message for one of the images:

Traceback (most recent call last):
File "data_preparation.py", line 48, in
data_prepare([os.path.join(opt.data_root, folder) for folder in opt.img_folder], opt.mode)
File "data_preparation.py", line 28, in data_prepare
detect_68p(img_folder, lm_sess, input_op, output_op) # detect landmarks for images
File "D:\Code\Python\Deep3DFaceRecon_pytorch\util\detect_lm68.py", line 105, in detect_68p
draw_landmarks(img, landmark, os.path.join(vis_path, name))
File "D:\Code\Python\Deep3DFaceRecon_pytorch\util\detect_lm68.py", line 29, in draw_landmarks
:] = np.array([0, 0, 255])
IndexError: index 432 is out of bounds for axis 0 with size 432

I changed corresponding index "img.shape[0]-landmark[i, 1]+j" to "img.shape[0]-1-landmark[i, 1]+j". And the above error didn't occur for the next try. I'm not sure if that's what you really meant.

BTW, thanks for sharing your work!

Hello,
I had such bug when I tried to inference some examples using test.py. The location of the bug is in nvdiffrast.py, line 57:

if self.glctx is None:
    self.glctx = dr.RasterizeGLContext(device=device)
    print("create glctx on device cuda:%d"%device.index)

My environment is Ubuntu 18.04, torch=1.6.0, CUDA=11.4, Driver Version=470.57.02

您好，我注意到您源码里训练时用的BFM model是经过转化以后只有35709个点的BFM model

我根据'BFM_exp_idx.mat'从原始BFM09（有53490个顶点）中提取了53215个点并且与“Exp_Pca.bin"中的表情基进行了整合，最终得到了'bfm_53215.mat'（相对于您用的35709个点的模型，增加了脖子和耳朵，验证过其拓扑结构也没有问题）。

我注意到您在根据回归出来的coeff来重建出个性化人脸模型时，进行了recenter，具体如下（详情可见/model/bfm.py)：

想问若我想使用我得到的'bfm_53215.mat'来重建出含有脖子和耳朵的个性化人脸模型，是否也需要recenter？

若recenter以后，在相同的coeff情况下，得到的含有脖子和耳朵的人脸模型再去掉脖子和耳朵以后 跟 直接将coeff作用于35709个点的BFM model得到的个性化人脸模型，他们的三维点坐标是不一样的。我个人认为的原因是在recenter是求平均值时，因为bfm_53215的meanshape的平均值与bfm_35709的meanshape的平均值不同。

After copying all requested resources and running

# get reconstruction results of example images
! python test.py --name=my_model--epoch=20 --img_folder=./datasets/examples

I get this error:

/usr/local/lib/python3.7/dist-packages/kornia/augmentation/augmentation.py:1875: DeprecationWarning: GaussianBlur is no longer maintained and will be removed from the future versions. Please use RandomGaussianBlur instead.
  category=DeprecationWarning,
/usr/local/lib/python3.7/dist-packages/torch/utils/cpp_extension.py:4: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
----------------- Options ---------------
                add_image: True                          
               bfm_folder: BFM                           
                bfm_model: BFM_model_front.mat           
                 camera_d: 10.0                          
                   center: 112.0                         
          checkpoints_dir: ./checkpoints                 
             dataset_mode: None                          
                 ddp_port: 12355                         
        display_per_batch: True                          
                    epoch: latest                        
          eval_batch_nums: inf                           
                    focal: 1015.0                        
                  gpu_ids: 0                             
               img_folder: ./datasets/examples           	[default: examples]
                init_path: checkpoints/init_model/resnet50-0676ba61.pth
                  isTrain: False                         	[default: None]
                    model: facerecon                     
                     name: my_model--epoch=20            	[default: face_recon]
                net_recon: resnet50                      
                    phase: test                          
                   suffix:                               
                  use_ddp: False                         	[default: True]
              use_last_fc: False                         
                  verbose: False                         
           vis_batch_nums: 1                             
               world_size: 1                             
                    z_far: 15.0                          
                   z_near: 5.0                           
----------------- End -------------------
Transfer BFM09 to BFM_model_front......
Traceback (most recent call last):
  File "test.py", line 72, in <module>
    main(0, opt,opt.img_folder)
  File "test.py", line 41, in main
    model = create_model(opt)
  File "/content/Deep3DFaceRecon_pytorch/models/__init__.py", line 65, in create_model
    instance = model(opt)
  File "/content/Deep3DFaceRecon_pytorch/models/facerecon_model.py", line 95, in __init__
    is_train=self.isTrain, default_name=opt.bfm_model
  File "/content/Deep3DFaceRecon_pytorch/models/bfm.py", line 40, in __init__
    transferBFM09(bfm_folder)
  File "/content/Deep3DFaceRecon_pytorch/util/load_mats.py", line 34, in transferBFM09
    original_BFM = loadmat(osp.join(bfm_folder, '01_MorphableModel.mat'))
  File "/usr/local/lib/python3.7/dist-packages/scipy/io/matlab/mio.py", line 218, in loadmat
    matfile_dict = MR.get_variables(variable_names)
  File "/usr/local/lib/python3.7/dist-packages/scipy/io/matlab/mio5.py", line 292, in get_variables
    res = self.read_var_array(hdr, process)
  File "/usr/local/lib/python3.7/dist-packages/scipy/io/matlab/mio5.py", line 252, in read_var_array
    return self._matrix_reader.array_from_header(header, process)
  File "mio5_utils.pyx", line 673, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
  File "mio5_utils.pyx", line 703, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
  File "mio5_utils.pyx", line 777, in scipy.io.matlab.mio5_utils.VarReader5.read_real_complex
  File "mio5_utils.pyx", line 450, in scipy.io.matlab.mio5_utils.VarReader5.read_numeric
  File "mio5_utils.pyx", line 355, in scipy.io.matlab.mio5_utils.VarReader5.read_element
  File "streams.pyx", line 200, in scipy.io.matlab.streams.ZlibInputStream.read_string
  File "streams.pyx", line 193, in scipy.io.matlab.streams.ZlibInputStream.read_into
OSError: could not read bytes

Hi there, I am new to this stuff... So it would be nice if you could guide me on how to save the texture information of the mesh as png file

Hi, I have ran the demo with nvidia driver 450.102.04. However, I failed when I use other environment with nvidia driver 450.80.02. Could you tell me your nvidia driver version?

How can I render straight face image from .obj?

When trying to run the examples

! python test.py --name=my_model--epoch=20 --img_folder=./datasets/examples

Full stack trace

Traceback (most recent call last):
  File "test.py", line 72, in <module>
    main(0, opt,opt.img_folder)
  File "test.py", line 41, in main
    model = create_model(opt)
  File "/content/Deep3DFaceRecon_pytorch/models/__init__.py", line 65, in create_model
    instance = model(opt)
  File "/content/Deep3DFaceRecon_pytorch/models/facerecon_model.py", line 95, in __init__
    is_train=self.isTrain, default_name=opt.bfm_model
  File "/content/Deep3DFaceRecon_pytorch/models/bfm.py", line 40, in __init__
    transferBFM09(bfm_folder)
  File "/content/Deep3DFaceRecon_pytorch/util/load_mats.py", line 34, in transferBFM09
    original_BFM = loadmat(osp.join(bfm_folder, '01_MorphableModel.mat'))
  File "/usr/local/lib/python3.7/dist-packages/scipy/io/matlab/mio.py", line 216, in loadmat
    with _open_file_context(file_name, appendmat) as f:
  File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.7/dist-packages/scipy/io/matlab/mio.py", line 19, in _open_file_context
    f, opened = _open_file(file_like, appendmat, mode)
  File "/usr/local/lib/python3.7/dist-packages/scipy/io/matlab/mio.py", line 45, in _open_file
    return open(file_like, mode), True
FileNotFoundError: [Errno 2] No such file or directory: 'BFM/01_MorphableModel.mat'

I have copied files in BFM folder and model to checkpoints folders:

Thank you for your amazing job~ Would you have plan to release the C-Net model weight or training script?

Hi, my environment is ubuntu18.04,cuda10.1, nvidia driver 430.46 and torch1.6. I encountered the errorkilled at 60 line of nvdiffrast.py ? I have install all dependence such as libglvnd0. Is it because of the low version of nvidia driver？

Hi, I wonder how can I render a video with preset camera motion? Just like what you did in the example.gif.

Also, is there a way to extract texture/material file from the obj file?

Thanks in advance!

Hi! Thanks for sharing your code, it is a great work! But there are something confusing me, could you please give some explanation? The first one is the rescale factor for aligning images. Why it is chosen to be 102? Can I change it? Another one is the similarity_Lm3D_all.mat file. I'd like to know how did you get these 3d landmarks in this file. Thanks again!

Hi @YuDeng , Thank you for sharing your great work.

How can I do model inferencing for the multi-image case as there is no C-Net available yet? My dataset has three images for each person and was taken from frontal and side.

You mentioned we could use the average shape for the multi-image case in #17 . I'm new to this area. Could you please expand on this?

If you could provide some example code would be much appreciated.

Thanks

Hi, thanks for your job. When I use train.py, I encounter some problems:

ImportError: No module named 'nvdiffrast_plugin' happen, I first set docker with cuda10 cudnn7, Then I deploy by your environment.yml.
I found a similar problem NVlabs/nvdiffrast#46

After regitered on Morphface, they sent me links of h5 files below, and they have different keys with 01_MorphableModel.mat, so can you help me?

Hi there,

Can you please tell me how I can include neck and ears in the generated model? I am assuming that I need to made some changes in the transferBFM09 method of util/load_mats.py, but any changes to index_exp (lines 61-62) is causing runtime errors in a number of other files.

Any help would be much appreciated

Hi,
Thanks for your great work.
I am confusing about the "ICP with isotropic scale for alignment" as mentioned in the paper.
It seems no much public code for the evaluation of the reconstruction error for the MICC Florence dataset.
Can you share the your implementation?

Any help would be appreciated!

Thank you for putting together such a tight implementation. I am trying to train from scratch, but keep running into the following error.

DeprecationWarning: GaussianBlur is no longer maintained and will be removed from the future versions. Please use RandomGaussianBlur instead.
  category=DeprecationWarning,
----------------- Options ---------------
                add_image: True
               batch_size: 32
           batch_size_val: 32
               bfm_folder: BFM
                bfm_model: BFM_model_front.mat
                 camera_d: 10.0
                   center: 112.0
          checkpoints_dir: ./checkpoints
           continue_train: False
                data_root: ./
             dataset_mode: flist
                 ddp_port: 12355
             display_freq: 1000
        display_per_batch: True
                    epoch: latest
              epoch_count: 1
          eval_batch_nums: inf
          evaluation_freq: 5000
                    flist: datalist/train/masks.txt
                flist_val: datalist/val/masks.txt
                    focal: 1015.0
                  gpu_ids: 0
                init_path: checkpoints/init_model/resnet50-0676ba61.pth
                  isTrain: True                                 [default: None]
                       lr: 0.0001
          lr_decay_epochs: 10
                lr_policy: step
         max_dataset_size: inf
                    model: facerecon
                 n_epochs: 20
                     name: sample_train                        [default: face_recon]
                net_recog: r50
           net_recog_path: checkpoints/recog_model/ms1mv3_arcface_r50_fp16/backbone.pth
                net_recon: resnet50
              num_threads: 4
                    phase: train
               preprocess: shift_scale_rot_flip
          pretrained_name: None
               print_freq: 100
                rot_angle: 10.0
             save_by_iter: False
          save_epoch_freq: 1
         save_latest_freq: 5000
              scale_delta: 0.1
           serial_batches: False
               shift_pixs: 10.0
                   suffix:
                  use_aug: True
            use_crop_face: True
                  use_ddp: False                                [default: True]
              use_last_fc: False
             use_predef_M: False
                  verbose: False
           vis_batch_nums: 1
                  w_color: 1.92
                    w_exp: 0.8
                   w_feat: 0.2
                  w_gamma: 10.0
                     w_id: 1.0
                     w_lm: 0.0016
                  w_reflc: 5.0
                    w_reg: 0.0003
                    w_tex: 0.017
               world_size: 1
                    z_far: 15.0
                   z_near: 5.0
----------------- End -------------------
rank 0 train dataset [FlistDataset] was created
rank 0 val dataset [FlistDataset] was created
loading init net_recon resnet50 from checkpoints/init_model/resnet50-0676ba61.pth
loading pretrained net_recog r50 from checkpoints/recog_model/ms1mv3_arcface_r50_fp16/backbone.pth
model [FaceReconModel] was created
The batch number of training images = 1
,             the batch number of validation images = 0
---------- Networks initialized -------------
[Network net_recon] Total number of parameters : 24.035 M
-----------------------------------------------
./checkpoints/sample_train/results
create rasterizer on device cuda:0
Traceback (most recent call last):
  File "train.py", line 166, in <module>
    main(0, world_size, train_opt)
  File "train.py", line 79, in main
    model.optimize_parameters()   # calculate loss functions, get gradients, update network weights
  File "/home/arstropica/dev/repos/Deep3DFaceRecon/models/facerecon_model.py", line 178, in optimize_parameters
    self.compute_losses()
  File "/home/arstropica/dev/repos/Deep3DFaceRecon/models/facerecon_model.py", line 152, in compute_losses
    pred_feat = self.net_recog(self.pred_face, trans_m)
  File "/home/arstropica/miniconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/arstropica/dev/repos/Deep3DFaceRecon/models/networks.py", line 122, in forward
    image = self.preprocess(resize_n_crop(image, M, self.input_size))
  File "/home/arstropica/dev/repos/Deep3DFaceRecon/models/networks.py", line 24, in resize_n_crop
    return warp_affine(image, M, dsize=(dsize, dsize))
  File "/home/arstropica/miniconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/kornia/geometry/transform/imgwarp.py", line 197, in warp_affine
    grid = F.affine_grid(src_norm_trans_dst_norm[:, :2, :], [B, C, dsize[0], dsize[1]], align_corners=align_corners)
  File "/home/arstropica/miniconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/nn/functional.py", line 3483, in affine_grid
    return torch.affine_grid_generator(theta, size, align_corners)
RuntimeError: invalid argument 7: equal number of batches expected at /opt/conda/conda-bld/pytorch_1603729128610/work/aten/src/THC/generic/THCTensorMathBlas.cu:32

I have preprocessed training & validation datasets, each with a relatively small number of images. But I am not sure what the error pertains to.
Any help appreciated.

Hi, thanks for your great work!
Following the installation instructions, I changed three servers, and the same error occurred. Is there any problem with this installation method or I haven't noticed anything?

Hello, I followed the instructions provided by the readme to debug the demo, and the following error occurred. How to solve this problem, can you give me some suggestions?

The platform is window10

想问一下是如何得到NoW challenge里人脸图片的5 lmk，是人脸图片直接用"How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks"的3D lmk 检测还是用了NoW里提供的bbox直接裁剪成256x256?参加挑战的模型是公开的epoch_20.pth吗？生成的obj文件里顶点的精度是np.float32吗？

Or how to output the obj file with straight front face ?

I don't know how to solve the problem,please help me!!!

(deep3d_pytorch) PS D:\Deep3DFaceRecon_pytorch> python test.py --name=CelebA --epoch=20 --img_folder=./datasets/examples
D:\anaconda3\envs\deep3d_pytorch\lib\site-packages\kornia\augmentation\augmentation.py:1875: DeprecationWarning: GaussianBlur is no longer maintained and will be removed from the future versions. Please use RandomGaussianBlur instead.
category=DeprecationWarning,
D:\anaconda3\envs\deep3d_pytorch\lib\site-packages\torch\utils\cpp_extension.py:4: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
----------------- Options ---------------
add_image: True
bfm_folder: BFM
bfm_model: BFM_model_front.mat
camera_d: 10.0
center: 112.0
checkpoints_dir: ./checkpoints
dataset_mode: None
ddp_port: 12355
display_per_batch: True
epoch: 20 [default: latest]
eval_batch_nums: inf
focal: 1015.0
gpu_ids: 0
img_folder: ./datasets/examples [default: examples]
init_path: checkpoints/init_model/resnet50-0676ba61.pth
isTrain: False [default: None]
model: facerecon
name: CelebA [default: face_recon]
net_recon: resnet50
phase: test
suffix:
use_ddp: False [default: True]
use_last_fc: False
verbose: False
vis_batch_nums: 1
world_size: 1
z_far: 15.0
z_near: 5.0
----------------- End -------------------
model [FaceReconModel] was created
loading the model from ./checkpoints\CelebA\epoch_20.pth
0 ./datasets/examples\000002.jpg
1 ./datasets/examples\000006.jpg
2 ./datasets/examples\000007.jpg
3 ./datasets/examples\000031.jpg
4 ./datasets/examples\000033.jpg
5 ./datasets/examples\000037.jpg
6 ./datasets/examples\000050.jpg
7 ./datasets/examples\000055.jpg
8 ./datasets/examples\000114.jpg
9 ./datasets/examples\000125.jpg
10 ./datasets/examples\000126.jpg
11 ./datasets/examples\015259.jpg
12 ./datasets/examples\015270.jpg
13 ./datasets/examples\015309.jpg
14 ./datasets/examples\015310.jpg
15 ./datasets/examples\015316.jpg
16 ./datasets/examples\015384.jpg
17 ./datasets/examples\vd006.png
18 ./datasets/examples\vd025.png
19 ./datasets/examples\vd026.png
20 ./datasets/examples\vd034.png
21 ./datasets/examples\vd051.png
22 ./datasets/examples\vd070.png
23 ./datasets/examples\vd092.png
24 ./datasets/examples\vd102.png

I have upsampled the image in order to reduce aliasing. The resulting image became much smoother except for the eye part, where there is obvious noise:

Here's the code I've used to produce the middle image:

high_resolution_rast, _ = dr.rasterize(self.glctx, vertex_ndc.contiguous(), tri, resolution=[1120, 1120], ranges=ranges)
high_resolution_image, _ = dr.interpolate(feat, high_resolution_rast, tri)

sum_mask_mat = self.sum_mask_mat.to(device)
sum_image_mat = self.sum_image_mat.to(device)

mask_avg = high_resolution_image.sum(3)[:, :, :, None]
mask_avg[mask_avg > 0] = 1.0
mask_avg = mask_avg.permute(0, 3, 1, 2)

mask_avg = torch.nn.functional.conv2d(mask_avg, sum_mask_mat, stride=5, padding=2)
mask_avg[mask_avg == 0.0] = 1.0

conv_image = torch.nn.functional.conv2d(high_resolution_image.permute(0, 3, 1, 2), sum_image_mat, stride=5, padding=2, groups=3)
conv_image = conv_image.div(mask_avg)

Is there something wrong with my code?

I will greatly appreciate your help.

Hello,

impossible to download the file via https://faces.dmi.unibas.ch/bfm/main.php?nav=1-2&id=downloads because I don't receive the email.

Too bad to waste all this work for a simple email

Greate work. I'm having a problem on running the inference code.
Firstly, I meeted the problem of fatal error: EGL/egl.h: No such file or directory.
I installed a package by apt-get install libegl1-mesa-dev.
Then I met another problem
[F glutil.cpp:338] eglInitialize() failed
Aborted (core dumped).,
Do you have any ideas how to fix this problem?

My Linux Environment:
cudnn: 7.6
cuda: 10.2
python :3.6
pytorch:1.6.0

I can't understand the part in code and I find it difficult to do mannually.These are blender screen shots.

Hi, I noticed that the z_far and z_near is 5 and 15 in the pytorch version.However, the near_clip is 0.01 and far_clip is 50.0 in the tensorflow version. I wonder why there is such a difference？

Hello, I am using mtcnn and there is no corresponding txt file, but only a picture of the detection point is generated for me. Is my operation wrong?

I run mtcnn output results

According to the documentation, how should I map the monitoring points to a txt file? thanks

Thanks for your greate work! I'm a recruit for 3d reconstruction , and I want to evaluate my model in NoW challenge. I follow the guide from https://github.com/soubhiksanyal/now_evaluation

python check_predictions.py <predicted_mesh_path> <predicted_mesh_landmark_path> <gt_scan_path> <gt_lmk_path> 

There is a little doubt about the <predicted_mesh_landmark_path>. It looks like the file containing 3D coordinates of 7 key points. Can you give me some advice or guidance on obtaining the 3D coordinates of key points？ Thanks again!

raise EnvironmentError('CUDA_HOME environment variable is not set. '
OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

I am getting above error by following the install instruction, do you know why? thanks for the explaination

Dear Authors,

Thanks for sharing your code! I found in this repository the w_feat is set to 1.0, w_id=0.2, and w_tex=1.7e-2 while in your paper they are set to w_per=0.2, w_alpha=1.0 and w_gamma=1.7e-3 (this w_gamma is the w_tex in this repository). Could you please tell me why they are different from what's in your paper? Besides I found neither the settings in your paper nor the settings in this repository can give me a model as good as what you released. Could you please help me to find what's wrong? I used MTCNN to extract the 5 facial landmarks and followed your data preparation procedure to prepare the training data. For the images in 300W-LP and LS3D-W datasets, I loaded the ground truth 68 landmarks directly and used the method extract_5p() in this repository to extract the 5 facial landmarks. I didn't balance the pose and race distributions, will this influence much? Thank you very much for reading this issue!

Hi, it was very excellent work. I meet some errors when I test the examples on the Ubuntu18.08 or Ubuntu 20.04. In Ubuntu18.04 the torch was 1.6 and the torch version was 1.8 in Ubunbu20.04. Here is the information in the terminal: Thanks for your attention.

(deep3d_pytorch) heizhima@heizhima-Super-Server:~/PycharmProjects/WXL/Deep3DFaceRecon_pytorch$ python test.py --name=pretrained --epoch=20 --img_folder=./datasets/examples/
/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/kornia/augmentation/augmentation.py:1875: DeprecationWarning: GaussianBlur is no longer maintained and will be removed from the future versions. Please use RandomGaussianBlur instead.
  category=DeprecationWarning,
/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py:4: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
----------------- Options ---------------
                add_image: True                          
               bfm_folder: BFM                           
                bfm_model: BFM_model_front.mat           
                 camera_d: 10.0                          
                   center: 112.0                         
          checkpoints_dir: ./checkpoints                 
             dataset_mode: None                          
                 ddp_port: 12355                         
        display_per_batch: True                          
                    epoch: 20                                   [default: latest]
          eval_batch_nums: inf                           
                    focal: 1015.0                        
                  gpu_ids: 0                             
               img_folder: ./datasets/examples/                 [default: examples]
                init_path: checkpoints/init_model/resnet50-0676ba61.pth
                  isTrain: False                                [default: None]
                    model: facerecon                     
                     name: pretrained                           [default: face_recon]
                net_recon: resnet50                      
                    phase: test                          
                   suffix:                               
                  use_ddp: False                                [default: True]
              use_last_fc: False                         
                  verbose: False                         
           vis_batch_nums: 1                             
               world_size: 1                             
                    z_far: 15.0                          
                   z_near: 5.0                           
----------------- End -------------------
model [FaceReconModel] was created
loading the model from ./checkpoints/pretrained/epoch_20.pth
0 ./datasets/examples/000002.jpg
Traceback (most recent call last):
  File "/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1515, in _run_ninja_build
    env=env)
  File "/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 72, in <module>
    main(0, opt,opt.img_folder)
  File "test.py", line 62, in main
    model.test()           # run inference
  File "/home/heizhima/PycharmProjects/WXL/Deep3DFaceRecon_pytorch/models/base_model.py", line 162, in test
    self.forward()
  File "/home/heizhima/PycharmProjects/WXL/Deep3DFaceRecon_pytorch/models/facerecon_model.py", line 139, in forward
    self.pred_vertex, self.facemodel.face_buf, feat=self.pred_color)
  File "/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/heizhima/PycharmProjects/WXL/Deep3DFaceRecon_pytorch/util/nvdiffrast.py", line 57, in forward
    self.glctx = dr.RasterizeGLContext(device=device)
  File "/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/ops.py", line 151, in __init__
    self.cpp_wrapper = _get_plugin().RasterizeGLStateWrapper(output_db, mode == 'automatic', cuda_device_idx)
  File "/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/ops.py", line 84, in _get_plugin
    torch.utils.cpp_extension.load(name=plugin_name, sources=source_paths, extra_cflags=opts, extra_cuda_cflags=opts, extra_ldflags=ldflags, with_cuda=True, verbose=False)
  File "/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 974, in load
    keep_intermediates=keep_intermediates)
  File "/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1179, in _jit_compile
    with_cuda=with_cuda)
  File "/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1279, in _write_ninja_file_and_build_library
    error_prefix="Error building extension '{}'".format(name))
  File "/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1529, in _run_ninja_build
    raise RuntimeError(message)
RuntimeError: Error building extension 'nvdiffrast_plugin': [1/4] c++ -MMD -MF torch_rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda-10.2/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_rasterize.cpp -o torch_rasterize.o 
FAILED: torch_rasterize.o 
c++ -MMD -MF torch_rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda-10.2/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_rasterize.cpp -o torch_rasterize.o 
In file included from /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/../common/rasterize.h:42:0,
                 from /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_rasterize.cpp:12:
/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/../common/glutil.h:36:21: fatal error: EGL/egl.h: No such file or directory
compilation terminated.
[2/4] c++ -MMD -MF rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda-10.2/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cpp -o rasterize.o 
FAILED: rasterize.o 
c++ -MMD -MF rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda-10.2/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cpp -o rasterize.o 
In file included from /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.h:42:0,
                 from /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cpp:9:
/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.h:36:21: fatal error: EGL/egl.h: No such file or directory
compilation terminated.
[3/4] c++ -MMD -MF glutil.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda-10.2/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.cpp -o glutil.o 
FAILED: glutil.o 
c++ -MMD -MF glutil.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda-10.2/include -isystem /home/heizhima/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.cpp -o glutil.o 
In file included from /home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.cpp:14:0:
/home/heizhima/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.h:36:21: fatal error: EGL/egl.h: No such file or directory
compilation terminated.
ninja: build stopped: subcommand failed.

Thanks for sharing this amazing repo!

Recently as I was exploring the quirks in the code, I noticed that the 68 landmarks extracted using the given indices (https://github.com/sicxu/Deep3DFaceRecon_pytorch/blob/master/util/load_mats.py) line 94 do not match with the provided landmarks in the similarity_Lm3D_all.mat file. There is a small offset between them, is there a reason to have this offset. I can send a pic if you would like.

Hi， when I run “python test.py --name=<model_name> --epoch=20 --img_folder=./datasets/examples”, I meet a trouble.
the problem is list below:

Traceback (most recent call last):
File "/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1515, in _run_ninja_build
env=env)
File "/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "test.py", line 72, in
main(0, opt,opt.img_folder)
File "test.py", line 62, in main
model.test() # run inference
File "/home/datas/18sdf/Deep3DFaceRecon/Deep3DFaceRecon_pytorch/models/base_model.py", line 162, in test
self.forward()
File "/home/datas/18sdf/Deep3DFaceRecon/Deep3DFaceRecon_pytorch/models/facerecon_model.py", line 139, in forward
self.pred_vertex, self.facemodel.face_buf, feat=self.pred_color)
File "/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/datas/18sdf/Deep3DFaceRecon/Deep3DFaceRecon_pytorch/util/nvdiffrast.py", line 58, in forward
self.glctx = dr.RasterizeGLContext(device=device)
File "/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/ops.py", line 151, in init
self.cpp_wrapper = _get_plugin().RasterizeGLStateWrapper(output_db, mode == 'automatic', cuda_device_idx)
File "/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/ops.py", line 84, in _get_plugin
torch.utils.cpp_extension.load(name=plugin_name, sources=source_paths, extra_cflags=opts, extra_cuda_cflags=opts, extra_ldflags=ldflags, with_cuda=True, verbose=False)
File "/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 974, in load
keep_intermediates=keep_intermediates)
File "/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1179, in _jit_compile
with_cuda=with_cuda)
File "/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1279, in _write_ninja_file_and_build_library
error_prefix="Error building extension '{}'".format(name))
File "/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1529, in _run_ninja_build
raise RuntimeError(message)
RuntimeError: Error building extension 'nvdiffrast_plugin': [1/4] c++ -MMD -MF glutil.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /root/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.cpp -o glutil.o
FAILED: glutil.o
c++ -MMD -MF glutil.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /root/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.cpp -o glutil.o
In file included from /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.cpp:14:0:
/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.h:36:21: fatal error: EGL/egl.h: No such file or directory
compilation terminated.
[2/4] c++ -MMD -MF torch_rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /root/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_rasterize.cpp -o torch_rasterize.o
FAILED: torch_rasterize.o
c++ -MMD -MF torch_rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /root/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_rasterize.cpp -o torch_rasterize.o
In file included from /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/../common/rasterize.h:42:0,
from /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/torch_rasterize.cpp:12:
/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/torch/../common/glutil.h:36:21: fatal error: EGL/egl.h: No such file or directory
compilation terminated.
[3/4] c++ -MMD -MF rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /root/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cpp -o rasterize.o
FAILED: rasterize.o
c++ -MMD -MF rasterize.o.d -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /root/anaconda3/envs/deep3d_pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -DNVDR_TORCH -c /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cpp -o rasterize.o
In file included from /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.h:42:0,
from /root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/rasterize.cpp:9:
/root/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/nvdiffrast/common/glutil.h:36:21: fatal error: EGL/egl.h: No such file or directory
compilation terminated.
ninja: build stopped: subcommand failed.

Do U know how to solve this problem?

Can I use it by Windows system？

如何将生成的人脸贴回去

Hi,

Thanks for your great work! I was wondering is there a way to visualize the face mesh on 2D image without texture and color?

One good example is 3rd column of Figure 2 in your paper. The default output is 2nd column in Figure 2 and no direct output can be the 3rd column visualization.

Best,
Yiwu

The nvdiffrast renderer has a very restrictive license policy which prevents using this code for anything but research (unlike in the TF repo). Is there a plan to add a support for pytorch3d or any other alternative differentiable renderer with a more permissive license?

Hi there, This is an awesome repo and I love it. It seems that nvdiffrast is tightly integrated with cuda and I couldn't find a way to properly make it work on a CPU-only machine. So It would be nice if you could guide me on setting this up on a CPU-only machine.