Fitting a TurboSquid Head mesh to a FLAME topology about voca HOT 5 CLOSED

Hi, there are two steps you need to do, First, you need to register the scan with the FLAME model (for details you please see the paper). Then you need to undo the effects of pose (i.e. global orientation, head pose, jaw pose) and facial expression. After this process, the mesh is in zero pose (i.e. canonical FLAME pose). Then it can be used together with VOCA.

from voca.

Hi thank you for taking the time to answer.

But it still doesn't clarify how to take a non-FLAME mesh and convert it to FLAME topology.

You mention:
First, you need to register the scan with the FLAME model
My question relates to using a mesh which is not a scan but a 3D mesh modeled by a 3d artist.
In the paper, I can read: the authors provided us with a static mesh, to which we aligned the FLAME topology.
But how can you align the FLAME topology over a 3D model?

You also mention: Then you need to undo the effects of pose
A handmade 3D model should already by in a neutral pose, from my understanding it just need to have it's transform reset so that the mesh position overlays one of the FLAME ply meshes.
I think I can do it using Blender.

(fyi, my skill set is more about 3d animation then ML.)

from voca.

Has anyone got any hints regarding how to register a 3D mesh, modeled by a 3d artist, to FLAME topology?
IE a mesh purchased on TurboSquid.
I am sorry, I fail to find instructions in the paper.

from voca.

Thank you, please check out Section 4.2 of the FLAME paper on how to align a general scan that is not in FLAME topology. The basic idea is to replace the vertex to vertex distance from the fit_3D_mesh demo with a general point to surface distance and minimize this instead. This is often referred to non-rigid ICP.

Another option could be using the provided FLAME Blender plugin to output a mesh in FLAME topology that then can be animated by VOCA.

from voca.

We also just added some demo code to fit FLAME to a scan of arbitrary topology in the flame-fitting repo.

from voca.