What's the purpose of this processing on rotation matrix? about graphcmr HOT 4 CLOSED

HMR uses the axis-angle representation for the rotation matrix, i.e. they regress the 3-dimensional axis angle vector for each joint that is then converted to a 3x3 rotation matrix inside the SMPL model using the Rodrigues formula.

Here we are regressing the full 3x3 rotation matrix, but the regressed matrix won't necessarily be a valid rotation matrix. So we do an extra step of "projecting" it to the manifold of rotation matrices. The process that we are doing here is one of the possible ways of achieving this.

from graphcmr.

Thanks for your fast reply! It's very informative.
It sounds to me a natural choice to regress axis angle vector instead of rotation matrix. The axis angle vector is always valid, which seems superior than rotation matrix,because rotation matrix needs extra process to make it valid.
So, why you choose rotation matrix in your paper? Is it based on your prior assumption? Or you have tried both and found that rotation matrix is better?
I'm curious about the insight behind this choice. BTW, you really did a good job which seems so different from previous work!

from graphcmr.

Although it might seem simpler, regressing axis-angle representations (or Euler angles e.g.) is very challenging.

Other works (e.g. Pavlakos et al. CVPR2018) used axis angle but applied the loss on the rotation matrices instead of the axis angle representations, because distance in axis angle space is not always a good measure of the distance between the rotation matrices.

I tried both at some point and regressing the full 3x3 rotation matrix performed significantly better.

I would suggest reading this paper that discusses the problem of regressing 3D rotations in detail. Here they propose another 6D representation for 3x3 rotation that is continuous. It is hypothesized that it is easier to regress continuous functions with neural networks. I've used this 6D representation in other works and it seems to be working really well.

from graphcmr.

Thanks! I will read more about it!

from graphcmr.