Code Monkey home page Code Monkey logo

cuboidabstractionviaseg's Introduction

Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds

This repository is a PyTorch implementation for paper: Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds.
Kaizhi Yang, Xuejin Chen
SIGGRAPH 2021

Introduction

Representing complex 3D objects as simple geometric primitives, known as shape abstraction, is important for geometric modeling, structural analysis, and shape synthesis. In this paper, we propose an unsupervised shape abstraction method to map a point cloud into a compact cuboid representation. We jointly predict cuboid allocation as part segmentation and cuboid shapes and enforce the consistency between the segmentation and shape abstraction for self-learning. For the cuboid abstraction task, we transform the input point cloud into a set of parametric cuboids using a variational auto-encoder network. The segmentation network allocates each point into a cuboid considering the point-cuboid affinity. In addition, several novel losses are designed to jointly supervise the two branches in terms of geometric similarity and cuboid compactness.

Dependencies

  • Python 3.8.8.
  • CUDA 10.2.
  • PyTorch 1.5.1.
  • TensorboardX for visualization of the training process.

Dataset

We provide the ready-to-use datasets:

Dataset

Please unzip this file and set its path as the argument E_shapenet4096.

Pretrain models

Pretrain models

Training

python E_train.py --E_shapenet4096 PATH_TO_SHAPENET4096 --E_ckpts_folder PATH_TO_SAVE --D_datatype DATA_TYPE

Inference

python E_infer.py --E_shapenet4096 PATH_TO_SHAPENET4096 --E_ckpt_path DIRECTORY_TO_CHECKPOINT --checkpoint CHECKPOINT_NAME

Cite

Please cite our work if you find it useful:

@misc{yang2021unsupervised,
    title={Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds},
    author={Kaizhi Yang and Xuejin Chen},
    year={2021},
    eprint={2106.03437},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

License

MIT License

cuboidabstractionviaseg's People

Contributors

silenkzyoung avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

cuboidabstractionviaseg's Issues

about the evalution of segmentation

Thanks for the wonderful work! Could you share your code about the segmentation? How to align shapenet part labels to the segmentation results?

Some issues about cuboid compactness loss

Hi, Kai Zhi, great job. And I have a question about compactness loss after reading the paper. M is a hyperparameter and fixed, why should we use a loss function to reduce the number of parts?
Thanks again, looking forward to your reply.

Weights for `W_CST` - translation $t_m$ not included in loss

Hi @SilenKZYoung

I'm working on the code and trying to reconstruct your results. The pretrained checkpoints work fine and create the described results in your paper.

But I discovered, that the W_CST-Loss ist set to 0 per default in training.

parser.add_argument ('--W_CST', default = 0.00, type = float, help = 'CST loss weight, this loss is only for generation application')

Does it hurt the performance of the model? I could also not find any note of it in the published paper.

Thus the cuboid vector $p_m$ is only trained for rotation $r_m$, scale $s_m$ and existence $\delta_m$, and the translation $t_m$ does not appear in the loss.
This leads to the position of the cuboids to be randomly distributed in the 3D space, which is not the case with the pretrained checkpoints.

This design decision seems somewhat counterintuitive. Could you provide some ratio on this choice?

Releasing pre-trained models?

Hi -- awesome work!

I was just wondering if you were planning on releasing pre-trained models for any categories from ShapeNet? If you are, do you have any idea on when you might do so?

Thanks!

Confusion about the Table1 "Quantitative comparison of shape reconstruction performance by Chamfer Distance"

Hello, I encountered a little problem when trying to reproduce the reconstruction performance indicators in your paper. Since there is no such part in the project code, I ask a question here and hope to get your reply. Thank you very much!
Based on the pre-trained model, I uniformly sampled the generated geometry according to the method of the paper to obtain a predicted point cloud of 4096 points, and calculate the Chamfer Distance between it and the original point cloud (I used the pcu library to Calculation, link .)
The result obtained is roughly an order of magnitude away from yours (0.04 (in the paper) vs 0.004 (my experiment result)).
I Note that the point cloud is scale-transformed,as mentioned in the paper, "All shapes are pre-aligned and normalized to the unit scale", I am not sure if there are some other operations before calculating the CD distance.
What should I do to get the CD index in the paper? Thanks very much!

Attachment: The left side of the figure is my result after uniform sampling, and the right side is the original point cloud result. I calculated the CD between these two point clouds.image-20211129221534576

render

Hi, your results look very good, how did you render the final visual effect?

Batch Size - Out of Memory error? -> with newer PyTorch version

Hi @SilenKZYoung ,

I currently tried to evaluate your model. Unfortunately, the batch size of 32 does definitely not fit into 11GB of VRAM, not even 16. I could run the training only on a batch size of 8.

I used a RTX 2080Ti. Could you please tell me, how did you fit your Model on a GTX 1080Ti ?

I will try to get my hands on an GTX 1080Ti and try it once again.

ERROR MESSAGE:

Traceback (most recent call last):
  File "/clusterstorage/gkobsik/learning-relationships/./CuboidAbstractionViaSeg/E_train.py", line 344, in <module>
    main(args)
  File "/clusterstorage/gkobsik/learning-relationships/./CuboidAbstractionViaSeg/E_train.py", line 123, in main
    loss, loss_dict = loss_func(points, normals, outdict, None, hypara)
  File "/clusterstorage/gkobsik/anaconda3/envs/learning_relationships/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/clusterstorage/gkobsik/learning-relationships/CuboidAbstractionViaSeg/losses.py", line 62, in forward
    randn_dis = (torch.randn((batch_size,num_points)) * self.std).cuda().detach()
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.