raymondjiangkw / cogs Goto Github PK

View Code? Open in Web Editor NEW

56.0 3.0 5.0 130.05 MB

[SIGGRAPH'24] A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose

Home Page: https://raymondjiangkw.github.io/cogs.github.io/

License: Other

Python 6.01% Jupyter Notebook 92.80% Shell 0.06% C++ 0.22% Cuda 0.89% CMake 0.01% C 0.01%

gaussian-splatting sparse-view

cogs's Introduction

A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose
_{Official PyTorch implementation of the ACM SIGGRAPH 2024 paper}

A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose
Kaiwen Jiang, Yang Fu, Mukund Varma T, Yash Belhe, Xiaolong Wang, Hao Su, Ravi Ramamoorthi

Paper | Project | Video

Abstract: Novel view synthesis from a sparse set of input images is a challenging problem of great practical interest, especially when camera poses are absent or inaccurate. Direct optimization of camera poses and usage of estimated depths in neural radiance field algorithms usually do not produce good results because of the coupling between poses and depths, and inaccuracies in monocular depth estimation. In this paper, we leverage the recent 3D Gaussian splatting method to develop a novel construct-and-optimize method for sparse view synthesis without camera poses. Specifically, we construct a solution progressively by using monocular depth and projecting pixels back into the 3D world. During construction, we optimize the solution by detecting 2D correspondences between training views and the corresponding rendered images. We develop a unified differentiable pipeline for camera registration and adjustment of both camera poses and depths, followed by back-projection. We also introduce a novel notion of an expected surface in Gaussian splatting, which is critical to our optimization. These steps enable a coarse solution, which can then be low-pass filtered and refined using standard optimization methods. We demonstrate results on the Tanks and Temples and Static Hikes datasets with as few as three widely-spaced views, showing significantly better quality than competing methods, including those with approximate camera pose information. Moreover, our results improve with more views and outperform previous InstantNGP and Gaussian Splatting algorithms even when using half the dataset.

Requirements

Please refer to 3DGS for requirements of hardware.
We have done all the experiments on the Linux platform with NVIDIA 3080 GPUs.
Dependencies: see environment.yml for exact library dependencies. You can use the following commands with Miniconda3 to create and activate your Python environment. NOTICE: you need to install xformers and PyTorch3D, following their guidelines, manually.
- conda env create -f environment.yml
- conda activate cogs
- python -m ipykernel install --user --name=cogs
- ./install.sh

Getting started

Please refer to fcclip for downloading fcclip_cocopan.pth, and put it under submodules/fcclip.
Please refer to QuadTreeAttention for downloading indoor.ckpt and outdoor.ckpt, and put them under submodules/QuadTreeAttention.

Quick Demo

We provide a quick demo for you to play with.

Dataset Preparation

For Tanks & Temples Dataset, please refer to NoPe-NeRF for downloading. You will then need to run convert.py to estimate the intrinsics and ground-truth extrinsics for evaluation.
For Hiking Dataset, please refer to LocalRF for downloading. We truncate each scene to keep first 50 frames such that 3 training views can cover the whole scene. You will then need to run convert.py to estimate the intrinsics and ground-truth extrinsics for evaluation.
For your own dataset, you need to provide the intrinsics, and notice that our model assumes all views share the same intrinsics as well. You need to prepare the data under the following structure:

- images/                       -- Directory containing your images
- sparse/0
    - cameras.bin/cameras.txt   -- Intrinsics information in COLMAP format.

Afterwards, you need to run following commands to estimate monocular depths and semantic masks.

python preprocess_1_estimate_monocular_depth.py -s <path to the dataset>
python preprocess_2_estimate_semantic_mask.py -s <path to the dataset>

Training

To train a scene, after preprocessing, please use

python train.py -s <path to the dataset> --eval --num_images <number of trainig views>

Interface for available training options (you can find default values in the 'arguments/__init__.py'):

Options used for constructing a coarse solution:

Argument	Type	Description
`rotation_finetune_lr`	`float`	Learning rate for the quaternion of camera
`translation_finetune_lr`	`float`	Learning rate for the translation of camera
`scale_finetune_lr`	`float`	Learning rate for the scaling per primitive for aligning the monocular depth
`shift_finetune_lr`	`float`	Learning rate for the translation per primitive for aligning the monocular depth
`register_steps`	`int`	Number of optimization steps for registering the camera pose
`align_steps`	`int`	Number of optimization steps for adjusting both the camera pose and monocular depth

Options used for refinement:

Argument	Type	Description
`iterations`	`int`	Number of iterations for optimization. If this is changed, other relevant options should also be adjusted.
`depth_diff_tolerance`	`int`	Threshold of difference between aligned depth and rendered depth to be considered as unobserved regions
`farest_percent`	`float`	Percent of retained number of points after farest point down-sampling
`retain_percent`	`float`	Percent of retained number of points after uniform down-sampling
`add_frame_interval`	`int`	Interval of training views which are back-projected after registration and adjustment
`scale_and_shift_mode`	`'mask'` or `'whole'`	Align the monocular depth either per primitive based on mask, or as a whole

Other hyper-parameters should be self-explaining.

Testing

After a scene is trained, please first use

python eval.py -m <path to the saved model> --load_iteration <load iteration>

to estimate the extrinsics of testing views. If ground-truth extrinsics are provided, it will calculate the metrics of estimated extrinsics of training views as well.

After registering the testing views, please use render.py and metrics.py to evaluate the novel view synthesis performance.

Tips

As to training, you may need to tweak the hyper-parameters to adapt to different scenes for best performance. For example,

If the distance between consecutive training frames is relatively large, register_steps and align_steps are recommended to increase.
If the number of training views are relatively large (e.g., 30 or 60 frames), it is recommended to set add_frame_interval to be larger than 1, and decrease register_steps and align_steps to avoid alignment and back-projection for unnecessary frames and speed up the process. The number of iterations is also recommended to increase.
- E.g., we use add_frame_interval = 10, register_steps = 50, align_steps = 100, number of iterations = 30000 for the Horse scene with 60/120 training views.
If the training frame contains too many primitives, which makes the number of correspondence points located on each primitive relatively low, setting scale_and_shift_mode to whole, and depth_diff_tolerance to 1e9 may produce better results.
Our method focuses on scene-level sparse view synthesis. If you want to apply this algorithm to object-level sparse view synthesis, you may want to adjust the BASE_SCALING and BASE_SHFIT in preprocess_1_estimate_monocular_depth.py and scene/cameras.py. Besides, you may want to adjust the depth_diff_tolerance as well. For example, dividing their initial values by 10 should be helpful.

As to testing, we evaluate at both 3000 and 9000 iterations' checkpoints, and use the better one.

FAQs

I don't have intrinsics, and fail to estimate the intrinsics as well. Usually, the intrinsics come with your camera. You may use your camera to carefully and densely capture a scene, and use its estimated intrinsics. Meanwhile, our rasterizer supports differentiation through the intrinsic parameters. You may try to optimize the intrinsic parameters as well if you can manage to get a coarse estimation in the first place (Caution: We haven't tested it yet). Notice that, inheriting from the 3DGS, only perspective camera is supported, and the principal point must lie at the center, and there is no skew.
I want to use SIBR viewer provided by 3DGS to inspect the scene. Since we do not export registered cameras parameters into a format that can be read by the SIBR viewer (Maybe added later), a work-around is to first apply COLMAP which sees all the views to estimate the ground-truth extrinsics, and then ensure the extrinsics of the first frame in our pipeline aligns with that in COLMAP by Commenting L55-56, and Uncommenting L58-59 in ./utils/camera_utils.py. If you want to evaluate the pose metrics, it is also necessary to align the pose of the first frame as well.
I want to speed up the refinement process. Currently, our refinement process utilizes the same rasterizer as that in the construction phase. Since we just use its differentiability of camera parameters, it is possible to remove unnecessary parts (i.e., ray-sphere intersection) in the rasterizer during the refinement to alleviate control divergence and speed up the training.

Acknowledgement

This project is built upon 3DGS. We also utilize FC-CLIP, MariGold, and QuadTreeAttention. We thank authors for their great repos.

Citation

@article{COGS2024,
    title={A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose},
    author={Jiang, Kaiwen and Fu, Yang and Varma T, Mukund and Belhe, Yash and Wang, Xiaolong and Su, Hao and Ramamoorthi, Ravi},
    journal={SIGGRAPH},
    year={2024}
}

cogs's People

Stargazers

Watchers

Forkers

peterzs jetd1 jackzhousz cdchenlin

cogs's Issues

Q about COLMAP-Free 3DGS results

Hello, @RaymondJiangkw thank you for your excellent work! I have a question. The official code of the COLMAP-Free 3DGS paper does not seem to be open source yet. Did you reproduce the COLMAP-Free 3DGS code used in your paper yourself? If so, is it possible to share the code?

Provide interface documentation

Hi,
Could you provide a document of all the interfaces that can be used in train?

Rendered depths seem weird in demo.ipynb

Thanks for your wonderful work!

When I running the demo.ipynb, the output rendered depths seem weird and different from the original output of the notebook. Any idea? Btw, I create a new envonment following your guidelines in cuda 12.1.

Thank!

The training speed is very slow

I trained on the Tanks/Family dataset on a single 3060Ti GPU with the command:
python train.py -s ${source_path} --eval --num_images 100 -m ${model_path} --add_frame_interval 10 --register_steps 50 --align_steps 100 --iterations 30000
as suggested in Readme-Tips for a relatively large number of training views. However, the training speed after alignment is unusually slow, taking nearly 10 hours to complete. Do you know if this is normal?

[Question] How to backpropagate the gradient of the camera ?

Hiya! Thanks for your wonderful work! I would like to ask some questions about the the gradient of the camera (or the viewpoint).

I'm trying to backpropagate the gradient of the camera, when I use the vanilla diff-gaussian-rasterization and set the camera pose as nn.Parameter and use a adam to optimize them, I found the camera pose can not be update as they don't have grad. Does the vanilla rasterization can't backpropagate gradient of camreas? If so, could you kindly give me some advice about how to implement it ? I'm not very familiar with the CUDA programing.
If I use your surface-rasterization, can I only backpropagate the gradient of the camera by setting other vars to requires_grad_(False) such as Fov? (It seems that your rasterization backpropagate grad of many vars).

Thank you very much! @RaymondJiangkw

ERROR: Could not build wheels for diff_gaussian_rasterization, which is required to install pyproject.toml-based projects

Thank you for your excellent work. I have encountered some issues and would like to seek your help.

My package versions are:

Python: 3.11.5
PyTorch: 2.3.0
PyTorch-CUDA: 12.1

I encountered the following issues when installing the submodules (diff-gaussian-rasterization, diff-gaussian-rasterization-appr-surface), but there were no issues when installing diff-gaussian-rasterization through the 3DGS official repository. Could you please let me know if there is an official source for diff-gaussian-rasterization-appr-surface and how to resolve the following issues? Thank you very much!
`Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Processing ./submodules/diff-gaussian-rasterization
Preparing metadata (setup.py) ... done
Building wheels for collected packages: diff_gaussian_rasterization
Building wheel for diff_gaussian_rasterization (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [313 lines of output]
running bdist_wheel
running build
running build_py
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/diff_gaussian_rasterization
copying diff_gaussian_rasterization/init.py -> build/lib.linux-x86_64-cpython-311/diff_gaussian_rasterization
running build_ext
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/utils/cpp_extension.py:418: UserWarning: The detected CUDA version (12.2) has a minor version mismatch with to compile PyTorch (12.1). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/utils/cpp_extension.py:428: UserWarning: There are no g++ version bounds defined for CUDA version 12.2
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
building 'diff_gaussian_rasterization.C' extension
creating /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-311
creating /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-311/cuda_rasterizer
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/utils/cpp_extension.py:1967: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are
If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
warnings.warn(
Emitting ninja build file /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-311/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/5] /usr/local/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpyackward.o.d -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/torch/csrc/apinda3/envs/dust3r/lib/python3.11/site-packages/torch/include/TH -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/THC -I/usr/local/cuda-12.2/include -I/ho3r/include/python3.11 -c -c /data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/backward.cu -o /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.lia_rasterizer/backward.o -D__CUDA_NO_HALF_OPERATORS_ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --comp'"'' -I/data/fyj/COGS/submodules/diff-gaussian-rasterization/third_party/glm/ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-Di1011"' -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-311/cuda_rasterizer/backward.o
/usr/local/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-3d.o.d -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/torch/csrc/api/inclunvs/dust3r/lib/python3.11/site-packages/torch/include/TH -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/THC -I/usr/local/cuda-12.2/include -I/home/gpulude/python3.11 -c -c /data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/backward.cu -o /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x8erizer/backward.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-oI/data/fyj/COGS/submodules/diff-gaussian-rasterization/third_party/glm/ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND' -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
In file included from /data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/backward.cu:12:
/data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/backward.h:19:10: fatal error: glm/glm.hpp: No such file or directory
19 | #include <glm/glm.hpp>
| ^~~~~~~~~~~~~
compilation terminated.
[2/5] /usr/local/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpyorward.o.d -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/torch/csrc/api/da3/envs/dust3r/lib/python3.11/site-packages/torch/include/TH -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/THC -I/usr/local/cuda-12.2/include -I/homr/include/python3.11 -c -c /data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/forward.cu -o /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linurasterizer/forward.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compile' -I/data/fyj/COGS/submodules/diff-gaussian-rasterization/third_party/glm/ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYB11"' -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-311/cuda_rasterizer/forward.o
/usr/local/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-3.o.d -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/torch/csrc/api/includvs/dust3r/lib/python3.11/site-packages/torch/include/TH -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/THC -I/usr/local/cuda-12.2/include -I/home/gpu/ude/python3.11 -c -c /data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/forward.cu -o /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_izer/forward.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-optiata/fyj/COGS/submodules/diff-gaussian-rasterization/third_party/glm/ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
In file included from /data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/forward.cu:12:
/data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/forward.h:19:10: fatal error: glm/glm.hpp: No such file or directory
19 | #include <glm/glm.hpp>
| ^~~~~~~~~~~~~
compilation terminated.
[3/5] /usr/local/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpyasterizer_impl.o.d -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/torch/cu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/TH -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/THC -I/usr/local/cuda-12.2/includvs/dust3r/include/python3.11 -c -c /data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/rasterizer_impl.cu -o /data/fyj/COGS/submodules/diff-gaussian-rasterizationpython-311/cuda_rasterizer/rasterizer_impl.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-reloptions ''"'"'-fPIC'"'"'' -I/data/fyj/COGS/submodules/diff-gaussian-rasterization/third_party/glm/ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDD11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-311/cuda_rasterizer/rasterizer_impl.o
/usr/local/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-3zer_impl.o.d -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/torch/csrc/aponda3/envs/dust3r/lib/python3.11/site-packages/torch/include/TH -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/THC -I/usr/local/cuda-12.2/include -I/ht3r/include/python3.11 -c -c /data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/rasterizer_impl.cu -o /data/fyj/COGS/submodules/diff-gaussian-rasterization/build-311/cuda_rasterizer/rasterizer_impl.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-cs ''"'"'-fPIC'"'"'' -I/data/fyj/COGS/submodules/diff-gaussian-rasterization/third_party/glm/ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="ILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/data/fyj/COGS/submodules/diff-gaussian-rasterization/cuda_rasterizer/rasterizer_impl.cu:23:10: fatal error: glm/glm.hpp: No such file or directory
23 | #include <glm/glm.hpp>
| ^~~~~~~~~~~~~
compilation terminated.
[4/5] c++ -MMD -MF /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-311/ext.o.d -pthread -B /home/gpu/anaconda3/envs/dust3r/compiler_compall -fPIC -O2 -isystem /home/gpu/anaconda3/envs/dust3r/include -fPIC -O2 -isystem /home/gpu/anaconda3/envs/dust3r/include -fPIC -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/sitI/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/TH -dust3r/lib/python3.11/site-packages/torch/include/THC -I/usr/local/cuda-12.2/include -I/home/gpu/anaconda3/envs/dust3r/include/python3.11 -c -c /data/fyj/COGS/submodules/diff-gauss-o /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpython-311/ext.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDD11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
[5/5] /usr/local/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpyo.d -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/torch/csrc/api/includes/dust3r/lib/python3.11/site-packages/torch/include/TH -I/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/THC -I/usr/local/cuda-12.2/include -I/home/gpu/ade/python3.11 -c -c /data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu -o /data/fyj/COGS/submodules/diff-gaussian-rasterization/build/temp.linux-x86_64-cpyth-D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS_ --expt-relaxed-constexpr --compiler-options ''"'"'-fPICbmodules/diff-gaussian-rasterization/third_party/glm/ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxN_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu: In function ‘std::tuple<int, at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor> RasterizeGsor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, float, const at::Tensor&, const at::Tensor&, const at::Tensor&, float, float, i, int, const at::Tensor&, bool, bool)’:
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:134: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:180: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
|
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:258: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:298: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:449: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:492: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:535: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:574: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:649: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:89:685: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is dep.data_ptr() instead. [-Wdeprecated-declarations]
89 | rendered = CudaRasterizer::Rasterizer::forward(
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu: In function ‘std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor, at:rizeGaussiansBackwardCUDA(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, float, const at::Tensor&, const at::Tensoat, float, const at::Tensor&, const at::Tensor&, int, const at::Tensor&, const at::Tensor&, int, const at::Tensor&, const at::Tensor&, bool)’:
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:95: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:141: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:176: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
|
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:215: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:336: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:379: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
|
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:422: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:461: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:517: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is der.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:762: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:806: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
|
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:848: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:892: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:935: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:979: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:1021: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() isnsor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:1060: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() isnsor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:1103: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() isnsor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:163:1149: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() isnsor.data_ptr() instead. [-Wdeprecated-declarations]
163 | CudaRasterizer::Rasterizer::backward(P, degree, M, R,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu: In function ‘at::Tensor markVisible(at::Tensor&, at::Tensor&, at::Tensor&)’:
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:209:81: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
209 | CudaRasterizer::Rasterizer::markVisible(P,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:209:124: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
209 | CudaRasterizer::Rasterizer::markVisible(P,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:209:167: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is sor.data_ptr() instead. [-Wdeprecated-declarations]
209 | CudaRasterizer::Rasterizer::markVisible(P,
|
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
/data/fyj/COGS/submodules/diff-gaussian-rasterization/rasterize_points.cu:209:206: warning: ‘T* at::Tensor::data() const [with T = bool]’ is deprecated: Tensor.data() is dor.data_ptr() instead. [-Wdeprecated-declarations]
209 | CudaRasterizer::Rasterizer::markVisible(P,
| ^
/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/include/ATen/core/TensorBody.h:247:1: note: declared here
247 | T * data() const {
| ^ ~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 2107, in _run_ninja_build
subprocess.run(
File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/data/fyj/COGS/submodules/diff-gaussian-rasterization/setup.py", line 17, in <module>
      setup(
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/__init__.py", line 104, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 184, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
      dist.run_commands()
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 368, in run
      self.run_command("build")
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 91, in run
      _build_ext.run(self)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
      self.build_extensions()
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 870, in build_extensions
      build_ext.build_extensions(self)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
      self._build_extensions_serial()
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
      self.build_extension(ext)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 252, in build_extension
      _build_ext.build_extension(self, ext)
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 560, in build_extension
      objects = self.compiler.compile(
                ^^^^^^^^^^^^^^^^^^^^^^
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 683, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 1783, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/gpu/anaconda3/envs/dust3r/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 2123, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for diff_gaussian_rasterization
Running setup.py clean for diff_gaussian_rasterization
Failed to build diff_gaussian_rasterization
ERROR: Could not build wheels for diff_gaussian_rasterization, which is required to install pyproject.toml-based projects
`

An error occurred when estimating semantic mask

Hi, thanks for your nice work. I have meet an error when I used the preprocess_2_estimate_semantic_mask.py to generate the mask.

File "/home/ga/software/anaconda3/envs/cogs/lib/python3.11/site-packages/torch/nn/functional.py", line 5382, in multi_head_attention_forward
    raise RuntimeError(f"The shape of the 2D attn_mask is {attn_mask.shape}, but should be {correct_2d_size}.")
RuntimeError: The shape of the 2D attn_mask is torch.Size([77, 77]), but should be (128, 128).

How can I solve it?

How to remove the use of semantic_mask?

Hello, thank you for your excellent work. Could you please tell how to remove the use of semantic_mask? Thank you.

The render result are completely black images

Sorry to bother you, but I ran the following commands and got black images as the result:

python train.py -s Tanks/Museum --eval --num_images 6

python eval.py -m "output/2024-07-02_00-46-53_Museum_6/" --load_iteration 9000

python render.py -m "output/2024-07-02_00-46-53_Museum_6/" -s "COGS/Tanks/Museum/"

Could you please help me check what might be the problem? Thank you very much!

Custom DataSet

First of all, thanks for your fantastic work!

I have some questions about custom dataset.

Can i train a weight for new scene,so that apply it on the other data in the scene.For example, i want to generator view for ocean.I want to first train the network using the ocean wave dataset that I have already used. I then used this network to generate new perspectives on my other ocean wave datasets.

2.Can the new viewpoints I generate control the camera pose? Can I get the camera pose for the generated new view?

how get fcclip_cocopan.pth from fcclip ?

Rasterizer backward is too slow

Hi! Recently, I have apply your diff-gaussian-rasterization-appr-surface to a vanilla gaussian model, and use DifferentiableCamera for pose optimization. The initial point cloud has 880k points, the loss is the same loss as vanilla gaussian (l1 + dssim). However, I find the loss.backward() takes too much time, around 1.9s per step. And i find that in your demo, backward of 500k points only takes 50ms and 1m points takes 100ms. Do you have any ideas? Thanks!

cc @RaymondJiangkw

An error when estimating monocular depths

Nice Work! But when I try to run the command to estimate monocular depth, an error is raising.

OSError: Bingxin/Marigold does not appear to have a file named config.json.

I did install transformers==4.32.1 according to the requirements file. Can you give me any advice to solve it?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.