This is an official code for the paper "Dynamic Point Fields" (ICCV 2023 Oral).

In a nutshell: we propose to model dynamic surfaces with a point-based model, where the motion of a point over time is represented by an implicit deformation field. Working directly with points (rather than SDFs) allows us to easily incorporate various well-known deformation constraints, e.g. as-isometric-as-possible (AIAP). We showcase the usefulness of this approach for creating animatable avatars in complex clothing:

Please see Section 4.3 of the paper for more details.

We release our code as a bundle of Colab notebooks covering the main experiments and applications discussed in the paper.

Description	Link
Static Surface Reconstruction with Optimised Clouds
Learning Surface Deformations
Single Scan Avatar Animation
Loading and Evaluating Original Paper Results (Sections 4.1, Table 1)	TBD
Loading and Evaluating Original Paper Results (Sections 4.2, Table 2)	TBD

We begin this work by revisiting point clouds as surface modeling primitives, and demonstrate how surfaces of arbitrary complexity can be efficiently modeled as sets of 3D points with associated features, such as normal directions:

Compared to the state-of-the-art implicit models for 3D surface representations (NGLOD, NGP), optimised point cloud model offers better reconstruction quality on all metrics, while taking zero inference time thanks to its explicit nature:

The explicit 3D surface formulation allows us to use some classic constraints for learning deformations in 3D space. In this work, we use the as-isometric-as-possible regularisation to guide our learning, which enforces the preservation of distances between points in the canonical and deformed spaces:

Please pay attention to the preservation of fingers in the deformed cloud when optimising with the isometric loss:

In the case of dynamic humans, we can directly supervise our deformation network with the information on the dynamics of the vertices of the underlying minimally clothed human model :

We showcase the advantages of our approach for animating 3D humans in challenging clothing. Using the introduced isometric and guidance losses to drive the source 3D shape allows to perform a single scan animation. Using this technique, we can repose the canonical scan to highly challenging target poses:

If you find our work useful in your research, please consider citing:

@article{prokudin2023dynamic,
                  title={Dynamic Point Fields},
                  author={Prokudin, Sergey and Ma, Qianli and Raafat, Maxime and Valentin, Julien and Tang, Siyu},
                  journal={arXiv preprint arXiv:2304.02626},
                  year={2023}
                }

See the LICENSE file.