Code Monkey home page Code Monkey logo

sp-gan's Introduction

SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation (SIGGRAPH 2021)

This repository contains a Pytorch implementation of the paper:

SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation.
Ruihui Li, Xianzhi Li, Ka-Hei Hui, Chi-Wing Fu.

teaser

Dependencies

  • Python 3.6
  • CUDA 10.0.
  • PyTorch. Codes are tested with version 1.2.0
  • (Optional) TensorboardX for visualization of the training process.

Following is the suggested way to install these dependencies:

# Create a new conda environment
conda create -n PointAugment python=3.6
conda activate PointAugment

# Install pytorch (please refer to the commend in the official website)
conda install pytorch=1.2.0 torchvision cudatoolkit=10.0 -c pytorch -y

Usage

As a generation task, you can freely customize your own dataset, such as ShapeNet , SMPL, SMAL

Or directly download our prepared H5 Files from here.

To train a model on point clouds sampled from 3D shapes:

python train.py --choice Chair

Log files and network parameters will be saved to log folder in default.

For your reference, the pretrained models are also available from here

We provide various visulization function for shape interpolation, part interpolation, and so on.

python visual.py

Evaluation

Please ref to Latent-GAN, PointFlow, and PDGN

Citation

If you find our work useful in your research, please consider citing:

@article{li2021spgan,
  title={{SP-GAN}:Sphere-Guided 3D Shape Generation and Manipulation},
  author={Li, Ruihui and Li, Xianzhi and Hui, Ke-Hei and Fu, Chi-Wing},
  journal={ACM Transactions on Graphics (Proc. SIGGRAPH)},
  volume={40},
  number={4},
  year={2021},
  publisher={ACM}
}

sp-gan's People

Contributors

liruihui avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

sp-gan's Issues

Generated point clouds look different when using batches

Hi, thank you for releasing the code base!

I encountered a strange issue. When I generate point clouds, my results are different depending on if I do it sequentially, or as a batch. Here is a small example (running inside model_test.py):

with torch.no_grad():
    batch_size = 10

    # generate random latent matrices for 10 shapes
    noise = self.noise_generator(bs=batch_size)

    # First try passing noise through model one at a time (sequentially)
    x = self.sphere_generator(bs=1)
    pcs_sequential= np.zeros((batch_size, 2048, 3))
    for i, latent_matrix in enumerate(noise):
        out_pc = self.G(x, latent_matrix[None])
        out_pc = out_pc.transpose(2, 1)
        out_pc = out_pc.cpu().detach().numpy()
        out_pc = normalize_point_cloud(out_pc)
        pcs_sequential[i] = out_pc 
    
    for i in range(batch_size): # Render results
        pyplot_draw_point_cloud(pcs_sequential[i], output_filename=f'nobatch_{i}.png')

    # Now try running all noise through model (full batch) 
    x = self.sphere_generator(bs=batch_size)
    
    out_pc = self.G(x, noise)
    out_pc = out_pc.transpose(2,1)

    pcs_batching = out_pc.cpu().detach().numpy()
    pcs_batching = normalize_point_cloud(pcs_batching)
    print (pcs_sequential - pcs_batching) # <--- We should expect this to be all zeros
    
    for i in range(batch_size): # Render results
        pyplot_draw_point_cloud(pcs_batching[i], output_filename=f'batch_{i}.png')

Here the point clouds generated sequentially and the point clouds generated in a batch are different. When I compute pcs_sequential - pcs_batching the values are non-zero, and the rendered point clouds are visibly different.

Any help would be appreciated!

Train custom dataset

hello
I want to train sp-gan on my own point cloud data,what kind of dataset shoud I prepare?what is the data form of your h5 file?

SMPL and SMAL datasets

Hi, Thanks for your excellent work.
Could you please share the SMPL and SMAL datasets you used?
I would be very appreciate if you can help me.

About the visualization in the paper

Hi, Thanks for your excellent work and the source code.
I am looking for the method for visualize the point cloud as a mesh, but I meet some problem when applying the method found
on the internet due to the lack of normal. (e.g. Poisson, ball pivot)
And I find the decent result in the bottom of Fig.9 in your paper.

Is it possible to release the source code of this part (convert point cloud to mesh)?
It will be really helpful to me.

Thank you very much!

About the detail shape

Hi,thank you for your great work!
I have tried part regions editing by your code ,but it seems that the mask[512,1024...] can't find the correspond part shape,such as chair’s back, legs, etc.Did I have to find the points representing the chair's legs on the ball manually?How do you solve it ?
plot_part_shape_inte_detail_20221124-1854_4

about evaluation

thanks for release the training and visualization code, based on your advise I use the PointFlow/metrics to evaluate, but the CD and EMD both 1,the all reasults as follow picture ,I can‘t analyze why, could you give me further advise or could you release the test code? Tanks a lot.
Screenshot from 2022-04-14 16-23-59

About Evaluation

Hi, thanks for releasing the code.

During training, SP-GAN uses a different data normalization approach (instance-wise normalizing the points to fit a unit ball) comparing to prior works such as PointFlow (zero-mean per axis and unit-variance globally). How about during the evaluation, how is the data preprocessed?

Would you release the evaluation code, or share how you preprocess/post-process the point cloud before evaluation?

Thank you!

chamferdistcuda

Hi, thank you for the great code! I have some issues with the train.py file. This error appears on personal training data set once the h5 files have been created and read:

ImportError: cannot import name 'chamferdistcuda' from 'metrics.CD_EMD.cd' (unknown location)

in file SP-GAN-main\metrics\CD_EMD\cd\chamferdist\ChamferDistance.py

Thank you for your time!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.