ybyangjing / superquadric_parsing Goto Github PK

This repository contains the code that accompanies our CVPR 2019 paper Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids

In order to be able to use our code please make sure that you have the following packages installed

• PyTorch
• matplotlib
• numpy
• progress
• pyquaternion
• trimesh
• Rtree
• backports.functools_lru_cache
• Cython
• PIL
• mayavi (Please note that in case you have difficulties installing mayavi via pip, this link might also be useful)

Alternatively, you can also try to simply run the following and hopefully all the aforementioned packages will be automatically installed

pip install --user -r requirements.txt

Subsequently, you need to build the Cython module that implements the sampling on the surface of the superquadrics by simply running

python setup.py build_ext --inplace

Now you are ready to start playing with the code.

For evaluating a previously trained model, we provide the forward_pass.py script that allows you to predict and visualize the predicted shape of an object as superquadric surfaces using mayavi.

You can run it by simply typing

$ ./forward_pass.py ~/data/03001627/ /tmp/ --model_tag "dac4af24e2facd7d3000ca4b04fcd6ac" --n_primitives 18 --weight_file ../config/chair_T26AK2FES_model_699 --train_with_bernoulli --use_deformations --use_sq --dataset_type shapenet_v2

The script requires two mandatory arguments, the path to the directory that contains the dataset ~/data/03001627 and the path to a directory that will be used for saving the generated files /tmp. You should also provide (even if it is not mandatory) the path to the previously trained model via the --weight_file argument, as well as the tag of the model you want to reconstruct (--model_tag) and the type of the dataset you are using (--dataset_type). Note that you should provide the same arguments that you used when training the model, regarding the configuration of the geometric primitives (e.g number of primitives, whether or not to use superquadrics etc.).

If you ran the above command you should see something like:

$ ./forward_pass.py ~/data/03001627/ /tmp/ --model_tag "dac4af24e2facd7d3000ca4b04fcd6ac" --n_primitives 18 --weight_file ../config/chair_T26AK2FES_model_699 --train_with_bernoulli --use_deformations --use_sq --dataset_type shapenet_v2
No handlers could be found for logger "trimesh"
Running code on  cpu
Found 6778 'ShapeNetV2' models
1 models in total ...
R: [[-0.38369974  0.57926947  0.7191812 ]
 [-0.5103006   0.5160787  -0.6879362 ]
 [-0.7696545  -0.6309594   0.09758216]]
t: [[-0.01014123]
 [-0.04051362]
 [ 0.00080032]]
e: [1.3734075, 1.3150274]
K: [-0.33036062, 0.23313229]
R: [[-0.5362834   0.04193148 -0.8429957 ]
 [ 0.78693724  0.3859552  -0.4814232 ]
 [ 0.3051718  -0.92156404 -0.23997879]]
t: [[-0.18105912]
 [ 0.11244812]
 [ 0.09697253]]
e: [0.7390462, 1.157957]
...
R: [[ 0.9701394  -0.24186404 -0.01820319]
 [-0.24252652 -0.96831805 -0.059508  ]
 [-0.00323363  0.06214581 -0.99806195]]
t: [[-0.02585344]
 [-0.15909581]
 [-0.091534  ]]
e: [0.4002924, 0.40005013]
K: [0.41454253, 0.3229351]
0 3.581116e-08
1 0.99999857
2 0.99999917
3 3.7113843e-08
4 0.9999982
5 0.9999975
6 0.9999988
7 0.9999999
8 3.5458616e-08
9 3.721507e-08
10 3.711448e-08
11 3.9621053e-08
12 3.7611613e-08
13 0.9999976
14 0.99999905
15 0.9999981
16 0.9999982
17 0.99999785
Using 11 primitives out of 18

and a chair should appear.

To train a new network from scratch we provide the train_network.py script.

You can simply execute it by typing

$ ./train_network.py ~/data/03001627/ /tmp/ --use_sq --lr 1e-4 --n_primitives 20 --train_with_bernoulli --dataset_type shapenet_v2
Running code on  cpu
Save experiment statistics in 26EKQBNTG
Found 6778 'ShapeNetV2' models
6778 models in total ...
1000/1000 [==============================] - 6s 6ms/step
Epoch 1/150 |                                | 15/500 - loss: 0.0110078 - pcl_to_prim: 0.0024909 - prim_to_pcl: 0.0085169 - exp_n_prims: 9.8473

You need to specify the path to the directory containing the dataset directory as well as the path to save the generated files such as the trained models. Note that the script automatically generates a subfolder inside the specified output directory (in this case 26EKQBNTG), where it saves the trained models, three .txt files with the loss evolution and a .json file with the parameters used for the current experiment.

We also provide the visualize_sq.py script which allows you to quickly visualize superquadrics given a set of parameters as a set of points sampled on the surface of the superquadric surface.

You can simply execute it by providing your preferred shape and size parametrization as follows:

$ ./visualize_sq.py --shape 1.0,1.0 --size 0.25,0.25,0.25

Below are some example images of various superquadrics using different shape and size parameters

Contributions such as bug fixes, bug reports, suggestions etc. are more than welcome and should be submitted in the form of new issues and/or pull requests on Github.

Our code is released under the MIT license which practically allows anyone to do anything with it. MIT license found in the LICENSE file.

Below we list some papers that are relevant to the provided code.

Ours:

• Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids pdf

By Others:

• Learning Shape Abstractions by Assembling Volumetric Primitives pdf
• 3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks pdf
• Im2Struct: Recovering 3D Shape Structure From a Single RGB Image pdf

Below we also list some more papers that are more closely related to superquadrics

• Equal-Distance Sampling of Supercllipse Models pdf
• Revisiting Superquadric Fitting: A Numerically Stable Formulation link

If you found this work influential or helpful for your research, please consider citing

@Inproceedings{Paschalidou2019CVPR,
     title = {Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids},
     author = {Paschalidou, Despoina and Ulusoy, Ali Osman and Geiger, Andreas},
     booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
     year = {2019}
}