Output meshes with fine details about deca HOT 2 CLOSED

Hi Yu,

thank you for your interest in DECA.

Regarding the detail mesh, thank you for the suggestion. We just updated the code to also output the reconstructed mesh with applied displacements. Please pull and try again. To leverage DECA's animatable details, we recommend to follow the provided animation pipeline (see Section 4 of the paper for more information about DECA's animatable details).

Regarding the questions about the paper, there are several factors contributing to the qualitative and quantitative state-of-the-art performance of DECA, namely

1. A novel shape consistency loss (Equation 8) that encourages the shape of different images of the same subject to be the same.
2. Unlike most previous 3D face reconstruction methods, we use FLAME instead of Basel Face Model (BFM) to represent the coarse shape. FLAME's identity space is more expressive than BFM as demonstrated in Section 7.3 of the original FLAME paper.
3. The coarse DECA shape is trained from a large dataset of 2 Million images with a wide coverage of ethnicities (see Appendix A). This makes DECA robust to a large variation of ethnicities, face shapes, head poses, lighting conditions, etc. This is important for a high reconstruction quality.
4. The training data are automatically cleaned to avoid poor landmark labels (see Appendix A). We will release the landmark labels for the training data together with the training code in the future.
5. The different losses are carefully combined and weighted (see Appendix A). All losses, i.e. landmark loss (Eq. 5), photometric loss, identity loss (Eq. 7), and the shape consistency loss (Eq. 8) contribute to the quality and robustness of the reconstruction.
6. It is also worth mentioning the eye closure loss (Eq. 6). While it does not directly influence the reconstruction error on NoW [Sanyal et al. 2019], it makes the reconstructed eye region visually more appealing.

I hope that answers your questions

from deca.

Thanks for your detailed explanation. It really helps.

from deca.