What model was used to extract the body pose ? about audio2photoreal HOT 4 CLOSED

Great, thank you very much ! :)

from audio2photoreal.

Hi! Thanks for your interest in our work!

More about the 104 joint angles can be found by digging around the LBS model provided by: https://github.com/facebookresearch/audio2photoreal/blob/main/visualize/ca_body/utils/lbs.py

There's no direct code to extract it from say an image since this dataset was collected via mocap, but the pose definition can probably be found via the LinearBlendSkinning module in the above code.

And while I am not familiar with how to directly visualize the 3D skeletal points, you can get the untextured 3D mesh that the LBS generates via #26 . To do the 3D skeletal points, I believe you would have to fit a skeleton to the mesh and then visualize it.

Hope this bit of info helps!

from audio2photoreal.

Ok , Thank you !
I will dig into that.
So the choice of 104 joint angles is only linked to the mocap technology that was used. By the way can you tell which technology did you use (if it is not a private information obviously ) ?

Also an other question in you paper in the Appendix B. Method, you talk about "SO(3) representation for the joint angles", is there any article or paper that can explain this kind of representation of the humain skeleton ?

Thank you in advance,

from audio2photoreal.

Correct! The technology used is detailed here https://arxiv.org/abs/2105.10441. However it's something we cannot release fully.
And absolutely! When referring to SO(3) I'm saying they are euler angles. I find this guide useful for understanding what that means: https://www.mathworks.com/help/nav/ref/so3.html.
For the guide on how this is used for the human skeleton specifically, hope this article on wiki helps! https://en.wikipedia.org/wiki/Inverse_kinematics

from audio2photoreal.