jiangzhongshi / surfacenetworks Goto Github PK

View Code? Open in Web Editor NEW

91.0 7.0 20.0 65 KB

Source code for CVPR 2018 Oral paper "Surface Networks"

Python 97.79% Cuda 2.21%

cvpr2018 graph-neural-networks geometric-deep-learning

surfacenetworks's Introduction

Surface Networks

Ilya Kostrikov, Zhongshi Jiang, Daniele Panozzo, Denis Zorin, Joan Bruna

IEEE Conference on Computer Vision and Pattern Recognition CVPR 2018 (Oral)

Abstract

We study data-driven representations for three-dimensional triangle meshes, which are one of the prevalent objects used to represent 3D geometry. Recent works have developed models that exploit the intrinsic geometry of manifolds and graphs, namely the Graph Neural Networks (GNNs) and its spectral variants, which learn from the local metric tensor via the Laplacian operator.

Despite offering excellent sample complexity and built-in invariances, intrinsic geometry alone is invariant to isometric deformations, making it unsuitable for many applications. To overcome this limitation, we propose several upgrades to GNNs to leverage extrinsic differential geometry properties of three-dimensional surfaces, increasing its modeling power. In particular, we propose to exploit the Dirac operator, whose spectrum detects principal curvature directions --- this is in stark contrast with the classical Laplace operator, which directly measures mean curvature. We coin the resulting models Surface Networks (SN).

We prove that these models define shape representations that are stable to deformation and to discretization, and we demonstrate the efficiency and versatility of SNs on two challenging tasks: temporal prediction of mesh deformations under non-linear dynamics and generative models using a variational autoencoder framework with encoders/decoders given by SNs.

Full Text

To appear in the proceedings of CVPR 2018, the preprint pdf is here

Source Code

Source code is hosted on this GitHub repository. Instructions can be read from the argparse options.

Requirements

torch==1.0.0
scipy==1.0.0
numpy==1.14.2
matplotlib==2.2.2
plyfile==0.5
tqdm==4.26

(optional)Python bindings for libigl is used for geometry processing. It's not required for reproducing the experiment, but if you would like to recompute Laplacian, Dirac etc, it would be convenient to install through pip install git+https://github.com/jiangzhongshi/libigl@cluster-pyigl#egg=pyigl

Data

Spatio Tempro Prediction https://drive.google.com/file/d/1tpqN7vrbuwwDsJEuBbLFoY3o3Zwe2K8i/view?usp=sharing

Mesh MNIST https://drive.google.com/file/d/1TPuVgQK-vPqLnetgHnHpfPyAYGKuUzRv/view?usp=sharing

FAUST correspondence https://drive.google.com/file/d/1bLPL7eBt1IRPUiQYdDXpB_-uNOxj-9Uj/view?usp=sharing

License

Source code MPL2 licensed (FAQ).

Please cite our paper if it helps.

@inproceedings{kostrikov2018surface,
  title={Surface Networks},
  author={Kostrikov, Ilya and Jiang, Zhongshi and Panozzo, Daniele and Zorin, Denis and Burna Joan},
  booktitle={2018 {IEEE} Conference on Computer Vision and Pattern Recognition, {CVPR} 2018},
  year={2018}
}

surfacenetworks's People

Contributors

Stargazers

Watchers

Forkers

foreverdavid mygit007hub peterzhousz tommaoer hpp340 soinlovelin nussetorten jwgu leeyangg aaroncnu thomasbatard markytools jtpils lijian10086 chenxi840221 xujinglin lingfanyu wudangt hadisaadat

surfacenetworks's Issues

cupy.cuda.driver.CUDADriverError: CUDA_ERROR_NOT_INITIALIZED: initialization error

Thank you for the great code!
I have a problem. When i run the program on gpu, the output is as follows:

Load data Preprocess Dataset 100% (60000 of 60000) |####################| Elapsed Time: 0:00:20 Time: 0:00:20 100% (10000 of 10000) |####################| Elapsed Time: 0:00:03 Time: 0:00:03 Num parameters 90314 N/A% (0 of 937) | | Elapsed Time: 0:00:00 ETA: --:--:--Traceback (most recent call last): File "main.py", line 213, in <module> main() File "main.py", line 155, in main outputs = model(inputs, laplacian, mask) File "/home/jiang/work/ping/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__ result = self.forward(*input, **kwargs) File "/home/jiang/work/ping/SurfaceNetworks/src/mesh_mnist/models.py", line 43, in forward x = self._modules['rn{}'.format(i)](L, mask, x) File "/home/jiang/work/ping/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__ result = self.forward(*input, **kwargs) File "/home/jiang/work/ping/SurfaceNetworks/src/utils/utils_pt.py", line 125, in forward xs = [x, SparseBMMFunc()(L, x)] File "/home/jiang/work/ping/SurfaceNetworks/src/utils/cuda/sparse_bmm_func.py", line 39, in forward col_ind, col_ptr = batch_csr(matrix1._indices(), matrix1.size()) File "/home/jiang/work/ping/SurfaceNetworks/src/utils/cuda/batch_csr.py", line 39, in __call__ m.load(bytes(ptx.encode())) File "cupy/cuda/function.pyx", line 175, in cupy.cuda.function.Module.load File "cupy/cuda/function.pyx", line 176, in cupy.cuda.function.Module.load File "cupy/cuda/driver.pyx", line 141, in cupy.cuda.driver.moduleLoadData File "cupy/cuda/driver.pyx", line 72, in cupy.cuda.driver.check_status cupy.cuda.driver.CUDADriverError: CUDA_ERROR_NOT_INITIALIZED: initialization error 100% (937 of 937) |########################| Elapsed Time: 0:00:00 Time: 0:00:00

Is there any problem with my operation?

System information

Python version: 2.7.15
CUDA/cuDNN version: 10.0.130 / 7.5.0
GPU model and memory: Nvidia GeForce GTX 980
Nvidia driver version: 410.48
Linux Ubuntu 18.04

Question about evaluation

Hello again.

So, i've launched "as_rigid_as_possible" experiment, trained a model, it saved in the pts folder. But what's next? How to use it to draw something?

Update pyigl to new igl python binding

Issues running your code

Hello,

First, thank your for publishing your code here :)

I would like to run the neural network to test it, and try to adapt it to what I want to do.

I succeded to preprocess the data, by running add_laplacian.py on my meshes.

But now, I don't understand what to do next. I didn't find any documentation about it, except the first readme.

I tried to run train_4_normals.py, but I have issues for missing modules :

ModuleNotFoundError: No module named 'pyigl'

It seems it is related to what you wrote in the Readme :
(optional)Python bindings for libigl is used for geometry processing. It's not required for reproducing the experiment, but if you would like to recompute Laplacian, Dirac etc, it would be convenient to install through pip install git+https://github.com/jiangzhongshi/libigl@cluster-pyigl#egg=pyigl

However, this command doesn't work anymore, it can't find the correct branch.

Can you help me a bit, by telling what I'm doing wrong, or what I have to do ?

Best Regards,
Vlad

Issues running mesh_mnist/main.py

Hello and thanks for your code.

Don't know if i'll get any answers, but anyway.

I'm trying to launch main.py (with default settings) in mesh_mnist folder (i've downloaded mesh mnist of course from the google drive links). Training crashes on the line:
outputs = model(inputs, laplacian, mask)

If we'll look deeper, it crashes here:
xs = [x, torch.mm(L,x.view(-1, feat)).view(batch, node, feat)] (167 line of src/utils/utils_pt.py file).

It says The expanded size of the tensor (64) must match the existing size (503) at non-singleton dimension 0. Target sizes: [64, 64]. Tensor sizes: [503, 64]

It is torch.mm failing because L is (64,503,503) and x.view(-1, feat) is (32192, 64). By the way, as i know, torch.mm uses 2D tensors as input, no? Well, does anyone launched that and could advice something?

thank u ;)