uc-vision / smart-tree Goto Github PK

View Code? Open in Web Editor NEW

20.0 7.0 0.0 19.1 MB

Neural Medial Axis Approximation of Point Clouds for 3D Tree Skeletonization

License: MIT License

Shell 0.56% Python 99.44%

cloud point pointcloud skeleton skeletonization 3d point-clouds sparse-convolution structure

smart-tree's Introduction

💡🧠🤔 Smart-Tree 🌳🌲🌴

📝 Description:

This repository contains code from the paper "Smart-Tree: Neural Medial Axis Approximation of Point Clouds for 3D Tree Skeletonization".
The code provided is a deep-learning-based skeletonization method for point clouds.


Input point cloud.	Mesh output.	Skeleton output.

💾 Data:

Please follow instructions to download data from this link.

🔧 Installation:

First, make sure you have Conda installed, aswell as mamba. This will ensure the enviroment is created within a resonable timeframe.

To install smart-tree please use
bash create-env.sh
Then activate the environment using:
conda activate smart-tree

📈 Training:

To train the model open smart_tree/conf/training.yaml.

You will need to update (alternatively these can be overwritten with hydra):

training.dataset.json_path to the location of where your smart_tree/conf/tree-split.json is stored.
training.dataset.directory to the location of where you downloaded the data (you can choose whether to train on the data with foliage or without based on the directory you supply).

You can experiment with/adjust hyper-parameter settings too.

The model will then train using the following:

train-smart-tree

The best model weights and model will be stored in the generated outputs directory.

▶️ Inference / ☠️ Skeletonization:

We supply two different models with weights:

noble-elevator-58 contains branch/foliage segmentation.
peach-forest-65 is only trained on points from the branching structure.

If you wish to run smart-tree using your own weights you will need to update the model paths in the tree-dataset.yaml.

To run smart-tree use:
run-smart-tree +path=cloud_path
where cloud_path is the path of the point cloud you want to skeletonize.
Skeletonization-specific parameters can be adjusted within the smart_tree/conf/tree-dataset.yaml config.

📜 Citation:

Please use the following BibTeX entry to cite our work:

@inproceedings{dobbs2023smart,
  title={Smart-Tree: Neural Medial Axis Approximation of Point Clouds for 3D Tree Skeletonization},
  author={Dobbs, Harry and Batchelor, Oliver and Green, Richard and Atlas, James},
  booktitle={Iberian Conference on Pattern Recognition and Image Analysis},
  pages={351--362},
  year={2023},
  organization={Springer}
}

Star History

📥 Contact

Should you have any questions, comments or suggestions please use the following contact details: [email protected]

smart-tree's People

Contributors

Stargazers

Watchers

smart-tree's Issues

hydra.errors.MissingConfigException: Primary config module 'smart_tree..conf' not found when running 'run-smart-tree'

Hi there, I've encountered an error when running run-smart-tree +path=/home/lmga-titanx/smart-tree/apple/apple_1/xyz.npy, where /home/lmga-titanx/smart-tree/apple/apple_1/xyz.npy is the cloud path of one of the apple trees downloaded from the given data. Here is the traceback of the error:

Why don't i have permissions?

root@9759268o280b:/home/smart-tree-main# train-smart-tree
wandb: Currently logged in as: szz216. Use wandb login --relogin to force relogin
wandb: ERROR Error while calling W&B API: project not found (<Response [404]>)
Problem at: /home/smart-tree-main/smart_tree/model/train.py 188 main
Error executing job with overrides: []
Traceback (most recent call last):
File "/home/smart-tree-main/smart_tree/model/train.py", line 188, in main
wandb.init(
File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1206, in init
raise e
File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1187, in init
run = wi.init()
File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 786, in init
raise error
wandb.errors.CommError: It appears that you do not have permission to access the requested resource. Please reach out to the project owner to grant you access. If you have the correct permissions, verify that there are no issues with your networking setup.(Error 404: Not Found)

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.

During the skeletonization process, The subgraph process stops at 11% with the following error:

Inference on the non -default GPU，such as "cuda:5"

Thank you very much for your amazing work, but I encountered some problems when doing inference with the existing model and parameters:

When I specify the number of the GPU, I always find that the program will run on the default GPU (cuda:0) Simultaneously load an identical model.
I often encounter program jams when inferencing. I am not sure whether it is related to the first point.

I would like to understand how long normal inferencing takes and if there is a way to solve my problem. Thank you again for your outstanding contribution to this work.

ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

Hi author!

I met this problem on my windows machine when I uesed bash create-env.sh to intall this packege.
Do you know how to solve it?
Thanks!
ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.
ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

How do you get Mesh output and Skeleton output from inference as shown in README.md?

Among the geometries given in the visualization window, I couldn't find one that has the skeleton and mesh output. Can you please guide me on how to obtain the skeleton and mesh output?

Encountered a problem when creating the environment

First of all, congratulation on such an amazing work! I've encountered an error when creating the conda environment from running the bash file bash create-env.sh , and I hope that you can provide me with some insights on how to resolve the issue. Thank you!

ERROR:

Looking for: ['python=3.8', 'mamba', 'pip', 'cudatoolkit-dev=11.7', 'pytorch::pytorch-cuda=11.7', 'pytorch::pytorch', 'rapidsai::rapids=23.02', 'rapidsai::cugraph=23.02', 'rapidsai::cudf=23.02']


Encountered problems while solving.
Problem: nothing provides __glibc >=2.17,<3.0.a0 needed by cudf-23.02.00-cuda_11_py38_g5ad4a85b9d_0
Problem: nothing provides __glibc >=2.17,<3.0.a0 needed by cugraph-23.02.00-cuda11_py310_g450c25b8_0
Problem: nothing provides __glibc >=2.17,<3.0.a0 needed by cuspatial-23.02.00-py38_g6fe3841_0

Conda info:

     active environment : base
    active env location : /home/lmga-titanx/anaconda3
            shell level : 1
       user config file : /home/lmga-titanx/.condarc
 populated config files : 
          conda version : 4.12.0
    conda-build version : 3.21.8
         python version : 3.9.12.final.0
       virtual packages : __cuda=11.4=0
                          __linux=5.4.0=0
                          __glibc=2.27=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/lmga-titanx/anaconda3  (writable)
      conda av data dir : /home/lmga-titanx/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/lmga-titanx/anaconda3/pkgs
                          /home/lmga-titanx/.conda/pkgs
       envs directories : /home/lmga-titanx/anaconda3/envs
                          /home/lmga-titanx/.conda/envs
               platform : linux-64
             user-agent : conda/4.12.0 requests/2.27.1 CPython/3.9.12 Linux/5.4.0-150-generic ubuntu/18.04.6 glibc/2.27
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

OS: Ubuntu 18.04
Python version: 3.9
CUDA Version: 11.4
NVIDIA Driver Version: 470.182.03
GCC version: 8.4

AttributeError: module 'torch._C' has no attribute '_cuda_customAllocator'

Hello author,
I have some questions for you.
After I installed my environment following bash commands, I am trying to infer my own data based on the existing model.But I am facing problem:
run-smart-tree +path= xxx

Can you please tell me how to solve this problem?

TypeError: 'type' object is not subscriptable

Hi, there is a new error with the latest version when running run-smart-tree.

ERROR:
File "/home/lmga-titanx/smart-tree/smart_tree/cli.py", line 24, in main
pipeline.process_cloud(Path(cfg.path))
File "/home/lmga-titanx/smart-tree/smart_tree/pipeline.py", line 68, in process_cloud
cloud = self.preprocessing(cloud)
File "/home/lmga-titanx/smart-tree/smart_tree/dataset/augmentations.py", line 117, in call
cloud = augmentation(cloud)
File "/home/lmga-titanx/smart-tree/smart_tree/dataset/augmentations.py", line 42, in call
centre, (x, y, z) = cloud.bbox
File "/home/lmga-titanx/anaconda3/envs/smart-tree/lib/python3.8/site-packages/typeguard/init.py", line 910, in wrapper
memo = _CallMemo(python_func, _localns, args=args, kwargs=kwargs)
File "/home/lmga-titanx/anaconda3/envs/smart-tree/lib/python3.8/site-packages/typeguard/init.py", line 129, in init
hints = get_type_hints(func, localns=frame_locals)
File "/home/lmga-titanx/anaconda3/envs/smart-tree/lib/python3.8/typing.py", line 1264, in get_type_hints
value = _eval_type(value, globalns, localns)
File "/home/lmga-titanx/anaconda3/envs/smart-tree/lib/python3.8/typing.py", line 270, in _eval_type
return t._evaluate(globalns, localns)
File "/home/lmga-titanx/anaconda3/envs/smart-tree/lib/python3.8/typing.py", line 518, in _evaluate
eval(self.forward_code, globalns, localns),
File "", line 1, in
TypeError: 'type' object is not subscriptable