This is a forked tutorial guide and the main codes will not be updated

This forked GitHub project is intented for folks who have little to know command-line knowledge and want to run Nerfstudio to create gaussian splats. If you have used Instant NGP, Nerfstudio, or other similar command-line based radiance field projects, most likely you have already installed some or all of the depedencies required for this project.

I created two walkthrough videos to compliment the tutorial (coming soon). You can watch them independently or with this project page as reference. Please follow my YT channel for additional updates. Now let's get gaussian splatting with Nerfstudio!

The section below is from the original GitHub page. Jump down to "Start Here" for the actual tutorial.

It’s as simple as plug and play with nerfstudio!

Nerfstudio provides a simple API that allows for a simplified end-to-end process of creating, training, and testing NeRFs. The library supports a more interpretable implementation of NeRFs by modularizing each component. With more modular NeRFs, we hope to create a more user-friendly experience in exploring the technology.

This is a contributor-friendly repo with the goal of building a community where users can more easily build upon each other's contributions. Nerfstudio initially launched as an opensource project by Berkeley students in KAIR lab at Berkeley AI Research (BAIR) in October 2022 as a part of a research project (paper). It is currently developed by Berkeley students and community contributors.

We are committed to providing learning resources to help you understand the basics of (if you're just getting started), and keep up-to-date with (if you're a seasoned veteran) all things NeRF. As researchers, we know just how hard it is to get onboarded with this next-gen technology. So we're here to help with tutorials, documentation, and more!

Have feature requests? Want to add your brand-spankin'-new NeRF model? Have a new dataset? We welcome contributions! Please do not hesitate to reach out to the nerfstudio team with any questions via Discord.

Have feedback? We'd love for you to fill out our Nerfstudio Feedback Form if you want to let us know who you are, why you are interested in Nerfstudio, or provide any feedback!

We hope nerfstudio enables you to build faster 🔨 learn together 📚 and contribute to our NeRF community 💖.

You must have an NVIDIA video card with CUDA installed on the system. This library has been tested with version 11.8 of CUDA. You can find more information about installing CUDA here

Install Git.

Install Visual Studio 2022. This must be done before installing CUDA. The necessary components are included in the Desktop Development with C++ workflow (also called C++ Build Tools in the BuildTools edition).

Nerfstudio requires python >= 3.8. We recommend using conda to manage dependencies. Make sure to install Conda before proceeding.

You will need to set up an environment on your PC to install dependencies and code. This next section will be completed primarily in command prompt. If you are new to command prompt, I suggest watching the installation video on my YouTube channel.

conda create --name nerfstudio -y python=3.8
conda activate nerfstudio
pip install --upgrade pip

Install PyTorch with CUDA (this repo has been tested with CUDA 11.7 and CUDA 11.8) and tiny-cuda-nn. cuda-toolkit is required for building tiny-cuda-nn.

You can check which version of CUDA Toolkit you have installed by entering nvcc --version into command prompt. Use the instructions below that match your version of CUDA Toolkit to install PyTorch.

For CUDA 11.7:

pip install torch==2.0.1+cu117 torchvision==0.15.2+cu117 --extra-index-url https://download.pytorch.org/whl/cu117

conda install -c "nvidia/label/cuda-11.7.1" cuda-toolkit
pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

For CUDA 11.8:

pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118

conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit
pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

You will need to build Nerfstudio from source to access guassian splatting.

git clone https://github.com/nerfstudio-project/nerfstudio.git
cd nerfstudio
pip install --upgrade pip setuptools
pip install -e .

Success! You have installed Nerfstudio!

The following will teach you how to train a gaussian splat scene with default settings. This is trained to 30,000 steps.

Open command prompt and start your Nerfstudio conda environment:

conda activate nerfstudio

path to the nerfstudio folder

cd nerfstudio

This this tutorial, we will use the open dataset provided by the Nerfstudio team. The following command will download an image set that is already prepared for training.

ns-download-data nerfstudio --capture-name=poster

This downloads a folder of data into: /nerfstudio/data/nerfstudio/poster. The result should look like this:

If you want to use your own data, check out the Using Custom Data section.

Next, you will initiate training of the gaussian splat scene.

ns-train gaussian-splatting --data data/nerfstudio/poster

If everything works, you should see training progress like the following:

Navigating to the link at the end of the terminal will load the webviewer. If you are running on a remote machine, you will need to port forward the websocket port (defaults to 7007).

Note: At the time of writing this guide, there is a bug where some people will see 0.0.0.0:7007 for the link. Replace the 0.0.0.0 with your IPv4 address. Here are instruction to find your IP address.

Note: The webviewer will slow down the performance of the model training.

It is possible to resume training a checkpoint by running

ns-train gaussian-splatting --data data/nerfstudio/poster --load-dir {outputs/.../nerfstudio_models}

You can launch any pre-trained model checkpoint into the viewer without initiating further training by running

ns-viewer --load-config {outputs/.../config.yml} --vis viewer_beta

This is recommended if you want to load a project you fully trained in command line and are coming back to later visualize and create video renders.

Once you have a gaussian splatting scene trained, you can either render out a video or export the gaussian splats ply file to view in other platforms such as Unity and UE5.

First we must create a path for the camera to follow. This can be done in the viewer under the "RENDER" tab. Orient your 3D view to the location where you wish the video to start, then press "Add Keyframe". This will set the first camera key frame. Continue to new viewpoints adding additional cameras to create the camera path. Nerfstudio provides other parameters to further refine your camera path. You can press "Preview Render" and then "Play" to preview your camera sequence.

Once satisfied, input a render name and then press "Generate Command" which will display a modal that contains the command needed to render the video. Ensure you copy and post this command into a notes doc as it's easy to lose once you close the viewer.

Kill the training job by pushing ctrl+C in the command prompt window.

Next, run the command you copied from the viewer to generate the video. A video will be rendered and it will display the location path.

Other video export options are available, learn more by running

ns-render --help

Currently, this must be performed with command line. The export section of the viewer does not function correctly with gaussin splats.

You can export the splat ply by running

ns-export gaussian-splat --load-config {outputs/.../config.yml} --output-dir {path/to/directory}

Learn about the export options by running

ns-export gaussian-splat --help

The gaussian splatting training model contains many parameters that can be changed. Use the --help command to see the full list of configuration options.

ns-train gaussian-splatting --help

Using an existing dataset is great, but likely you want to use your own data! Nerfstudio supports various methods for using your own data. Before it can be used in nerfstudio, the camera location and orientations must be determined and then converted into our format using ns-process-data. Nerfstudio relies on external tools for this, instructions and information can be found in the documentation.

And that's it for creating gaussian-splats with nerfstudio.

If you're interested in learning more on how to create your own pipelines, develop with the viewer, run benchmarks, and more, please check out some of the quicklinks below or visit our documentation directly.