Code Monkey home page Code Monkey logo

elasticbasisforspectralmatching's Introduction

We compute correspondences between shapes by computing a functional map using (projected) eigenfunctions of an elastic shell energy’s Hessian as a functional basis. These basis functions are sensitive to extrinsic features such as extremities and creases (left). Hence, using them in a functional map pipeline enables us to accurately align crease lines such as mouth, ears, and toes (right). Here, we visualize the resulting correspondence by a pullback of normals from the ferret (top) to the weasel (bottom). The (transferred) normal directions are mapped to colors as shown on the little sphere.

An Elastic Basis for Spectral Shape Correspondence

This code accompanies the SIGGRAPH 2023 conference paper An Elastic Basis for Spectral Shape Correspondence, by Florine Hartwig, Josua Sassen, Omri Azencot, Martin Rumpf, Mirela Ben-Chen

In this paper we develop a spectral non-isometric correspondence method that aligns extrinsic features using a functional map approach. We propose a novel crease-aware spectral basis derived from the Hessian of an elastic thin shell energy and describe the necessary adaptations for using non-orthogonal basis functions in the functional map framework.

Requirements

This code uses python bindings for an implementation of the Discrete Shell Energy available here

and the following packages

  • numpy
  • scipy.sparse
  • scipy.spatial
  • libigl
  • polyscope (for visualization)
  • tqdm
  • matplotlib

Demo Scripts

We provide two example scripts which show the basis functionality of the code. For comparison, our code also provides the option to use the eigenfunctions of the Laplace Beltrami operator as a basis for the functional map approach. In this case our method reduces to ZoomOut.

  • Cartoon.py script to compute correspondences between modified versions of a Homer and a Max-Planck bust model. The necessary data is stored in data/ . For creating the deformed mesh versions, we used an implementation of Computational caricaturization of surfaces by Sela, Matan, Yonathan Aflalo, and Ron Kimmel. Computer Vision and Image Understanding 141 (2015): 1-17.
  • CatLion.py script to compute correspondences between a cat and lion shape available at http://people.csail.mit.edu/sumner/research/deftransfer/data.html. We provide a shell script which will automatically download and store the necessary data (cat-reference.obj and lion-reference.obj).

For more details and theoretic background have a look at our paper. If you should have any questions, feel free to reach out to Florine Hartwig!

Attribution

Please cite our paper when using this code. You can use the following bibtex

@inproceedings{HaSaAz23,
  title={An Elastic Basis for Spectral Shape Correspondence},
  author={Hartwig, Florine and Sassen, Josua and Azencot, Omri and Rumpf, Martin and Ben-Chen, Mirela},
  booktitle={ACM SIGGRAPH 2023 Conference Proceedings},
  year={2023},
  publisher = {Association for Computing Machinery},
  doi = {10.1145/3588432.3591518}
}

elasticbasisforspectralmatching's People

Contributors

flrneha avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

peterzs bastianlb

elasticbasisforspectralmatching's Issues

SqrtmassA in the code

Dear Author,

Thanks for the well-organized code and well explained paper. However, I had one confusion in your code. In Solver.py, when we're computing the functional map from point correspondence, the code uses 'square root of diagonal mass matrix A' , the code is as follows:
C = np.linalg.pinv(self.sqrtmassA.dot(self.basisA[:, :k])).dot(
self.sqrtmassA.dot(P.T.dot(self.basisB[:, :k])))
While in the paper, this process is formulated as:
20751702160504_ pic

I would like to ask why are we using the squared root of diagonal mass matrix here, rather than the mass matrix computed from igl?

Thanks for the answer.

Confusion about the code implementation when solving C_init

Hello! Thanks so much for your detailed and clear code implementation for the theory in the paper. But I have one confusion when we solve the C_init in the solver.py:

C_init = np.linalg.lstsq(np.kron(G, sqrtA), np.reshape(sqrtA.dot(F.T), (-1), order='F'), rcond=None)[0]
C_init = np.reshape(C_init, (self.kmin, self.kmin), order='F')

I can find the corresponding derivation from formula (13) in your paper, namely:
1702249842123
Although this formula aims to solve the correspondence map, if we already know the initial correspondence relationship between ei and ej, we can treat this formula as solving the optimization of functional map C.

But I am confused how I can deduce F.T from Φ1,k+ M1-1ej (I take the latter one as an example)? Because I have tried to deduce that these two items are equal, but I failed. Because I will always get additional M2,k term from Φ1,k+ M1-1ej.

Could you help me solve this problem? Thank you so much for your attention and participation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.