Difference between generic FLAME models and VOCA-compatible FLAME models? about voca HOT 10 CLOSED

As a sanity check you can also take the OBJ and convert it to PLY with MeshLab, which does not change vertex orders by default.

from voca.

For VOCA, only the mesh topology matters and this is same for all FLAME models. Generating templates by randomly sampling the identity face space should work for all FLAME models. Only the eye blink demo works solely for FLAME2019, as it uses pre-computed expression parameters to edit the eyelids.
When fitting any FLAME model to a registered 3D mesh in FLAME topology with the TF_FLAME code, global rotation, pose (head pose, jaw pose) and expression parameters must be set to zero. Then you should be able to use the result as VOCA template.
If you considered setting all these parameters to zero before exporting the mesh, and the problem remains, can you please show some renderings of these errors?

from voca.

Thank you for the reply!

As mentioned in the edit of my original pose, I think the problem may actually be with exporting the file from Blender.

I am working with OBJ FLAME files, and need to convert to PLY to input into VOCA.
For this, I need to import the OBJ file and export as PLY from Blender. It seems this step might be causing the artefacts I mentioned, as using the FLAME template mesh provided in the demo, exporting this as an OBJ file and back to a PLY file in Blender (without any other changes), then inputting back to VOCA creates the same artefacts (where the original template worked perfectly). I've attached a video of the artefacts described:

Do you know if it may be some export settings in Blender causing this, or conversion from OBJ to PLY?

from voca.

I see, it seems your vertex order is somehow scrambled. Make sure that Blender exports the meshes without changing the vertex order. (This could be either when importing OBJ into Blender, which has an "keep vertex order" flag that must be checked, or when exporting the mesh)

from voca.

I see, thank you! I will try this now and let you know if it works!

from voca.

I was able to get it to work using MeshLab, exporting the original working template as OBJ, then back to PLY, to generate a working animation output.

I will now try with TF_FLAME 'fit to 3D mesh' and setting global pose to zero.

from voca.

How can I set the pose to zero after fitting FLAME to a 3D mesh in FLAME topology (or does the mesh originally need to be in zero-pose)?

from voca.

Zero out the parameters for global rotation and translation, jaw, head, and eyeball poses, and the expression vector after the fitting, basically keeping only the identity shape parameters non-zero. Then just output the result of the forward pass through the FLAME layer for these parameters. This will give you a mesh in "zero pose"

from voca.

Possibly a stupid question, but is there a simple way of setting all the parameters to zero? A specific file I could modify or run? I had a brief look but could not find anything from TF_FLAME. Thank you in advance!

from voca.

Managed to make everything work. I extracted all the translation, rotation, pose, shape and expression parameters from "fit_3D_mesh.py", then generated a neutral mesh using only the extracted shape parameter with all other parameters zero-ed out using the assign method in "sample_FLAME.py".

from voca.