Code Monkey home page Code Monkey logo

inerf's Introduction

iNeRF: Inverting Neural Radiance Fields for 6-DoF Pose Estimation

Implementation of Inverting Neural Radiance Fields for Pose Estimation using PyTorch.

Installation

To start, I recommend to create an environment using conda:

conda create -n inerf python=3.8
conda activate inerf

Clone the repository and install dependencies:

git clone https://github.com/salykovaa/inerf.git
cd inerf
pip install -r requirements.txt

How to use

To run the algorithm on Lego object

python run.py --config configs/lego.txt

If you want to store gif video of optimization process, set OVERLAY = True here

All other parameters such as batch size, sampling strategy, initial camera error you can adjust in corresponding config files.

To run the algorithm on the llff dataset, just download the "nerf_llff_data" folder from here and put the downloaded folder in the "data" folder.

All NeRF models were trained using this code https://github.com/yenchenlin/nerf-pytorch/

├── data 
│   ├── nerf_llff_data   
│   ├── nerf_synthetic  

Examples

Different sampling strategies

Left - random, in the middle - interest points, right - interest regions. Interest regions sampling strategy provides faster convergence and doesnt stick in a local minimum like interest points.

Citation

Kudos to the authors

@article{yen2020inerf,
  title={{iNeRF}: Inverting Neural Radiance Fields for Pose Estimation},
  author={Lin Yen-Chen and Pete Florence and Jonathan T. Barron and Alberto Rodriguez and Phillip Isola and Tsung-Yi Lin},
  year={2020},
  journal={arxiv arXiv:2012.05877},
}

Parts of the code were based on yenchenlin's NeRF implementation: https://github.com/yenchenlin/nerf-pytorch

@misc{lin2020nerfpytorch,
  title={NeRF-pytorch},
  author={Yen-Chen, Lin},
  howpublished={\url{https://github.com/yenchenlin/nerf-pytorch/}},
  year={2020}
}

inerf's People

Contributors

salykova avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

inerf's Issues

Requirements probelm(resolved)

I followed the requirements install opencv-python 4.2, the erro shows:

Traceback (most recent call last): File "run.py", line 221, in <module> run() File "run.py", line 87, in run POI = find_POI(obs_img_noised, DEBUG) # xy pixel coordinates of points of interest (N x 2) File "B:\code\inerf\utils.py", line 184, in find_POI sift = cv2.SIFT_create() AttributeError: module 'cv2.cv2' has no attribute 'SIFT_create'

then I tired 4.1 and 4.5, changed the opencv-python opencv-contrib-python opencv-python-headless to 4.5.5.64, problem solved.

inerf accuracy

Thanks for your excellent work!
After reading your code, I find your start_pose has a 10 degree rotation for the z-axis.
So could I know whether your code able to recover pose within 40 degrees of rotation, as mentioned in the paper? In other words, what is the approximate accuracy of your code's maximum recovery of rotation and translation?

hope for your reply!

How can I apply inerf on custom dataset

Hello,

Thanks for this awesome work. I already generate a mesh using Nerf but I would like to estimate the 6D position, but I don't understand on how can I apply your model on my mesh.
Thanks in advance

Questions about the code

The initial poses(start_pose)in the code seem to have been calculated from GroundTruth,I don't know if I'm getting this wrong.If it's based on the calculation, isn't it inappropriate.
I would appreciate it if you could kindly reply to me in your spare time.

Lie algebra problem

Thank you for your excellent work!

During my running process, I found that optimizing the pose does not necessarily satisfy the Lie algebra. What could be the reason for this?

1677849261660

how can I get the transforms.json

Hi author. Thanks for your great work!
I want to ask how can I get the transforms.json when I test my own nerf model and data trained in nerf-pytorch ( which you mentioned in Readme.md)
I have trained my nerf model. When I want to test in inerf, it comes that I don't know how to generate the json file just like lego or chair you provided in your program.
image
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.