This is the official implementation of Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attention.
Project Page | Arxiv | Weights | 
teaser.mp4
conda create -n Era3D python=3.9
conda activate Era3D
# torch
pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu118
# install xformers, download from https://download.pytorch.org/whl/cu118
pip install xformers-0.0.23.post1-cp39-cp39-manylinux2014_x86_64.whl
# for reconstruciton
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
pip install git+https://github.com/NVlabs/nvdiffrast
# other depedency
pip install -r requirement.txt
You can directly download the model from huggingface. You also can download the model in python script:
from huggingface_hub import snapshot_download
snapshot_download(repo_id="pengHTYX/MacLab-Era3D-512-6view")
- we generate multivew color and normal images by running test_mvdiffusion_unclip.py. For example,
python test_mvdiffusion_unclip.py --config configs/test_unclip-512-6view.yaml \
pretrained_model_name_or_path='pengHTYX/MacLab-Era3D-512-6view' \
validation_dataset.crop_size='420' \
validation_dataset.root_dir=examples \
seed=600 \
save_dir='mv_res' \
save_mode='rgb'
You can adjust the crop_size (400 or 420) and seed (42 or 600) to obtain best results for some cases.
-
Typically, we use
rembgto predict alpha channel. If it has artifact, try to use Clipdrop to remove the background. -
Instant-NSR Mesh Extraction
cd instant-nsr-pl
bash run.sh $GPU $CASE $OUTPUT_DIR
For example,
bash run.sh 0 A_bulldog_with_a_black_pirate_hat_rgba recon
The textured mesh will be saved in $OUTPUT_DIR.
We collect code from following projects. We thanks for the contributions from the open-source community!
diffusers
Wonder3D
Syncdreamer
Instant-nsr-pl
This project is under AGPL-3.0, so any downstream solution and products that include our codes or the pretrained model inside it should be open-sourced to comply with the AGPL conditions. If you have any questions about the usage of Era3D, please feel free to contact us.
If you find this codebase useful, please consider cite our work.
@article{li2024era3d,
title={Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attention},
author={Li, Peng and Liu, Yuan and Long, Xiaoxiao and Zhang, Feihu and Lin, Cheng and Li, Mengfei and Qi, Xingqun and Zhang, Shanghang and Luo, Wenhan and Tan, Ping and others},
journal={arXiv preprint arXiv:2405.11616},
year={2024}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent