对于特定训练数据，head motion和candidate image是否有必要？ about livespeechportraits HOT 3 CLOSED

If your data doesn't include head poses and upper body motions, e.g., MEAD, of course, predicting head pose is meaningless because actually there're no head poses exist in the dataset right (heads in all training frames are fixed)?

Meanwhile, if your dataset doesn't include shoulder motions (or on a very small scale), removing upper body motion features may work. But it also depends on how you cut and crop the face region, e.g., if all the training frames keep the shoulders almost in the same location, I think upper body motion is not needed because there is no ambiguity about the bottom part of the images, i.e., the shoulder keeps fixed. After all, the shoulder line is designed to remove the ambiguity in the training set (One could move his shoulder while keeping his head fixed).

Also, if the background of your dataset is something like a green screen and your camera is fixed, i.e., camera parameters are fixed for all training frames, I think the candidate image set could also be removed.

Finally, I think every design should be considered and checked in the experiments, right?

from livespeechportraits.

谢谢Dr. Lu详细的解答！
再请教训练过程中，每个形象的3分钟视频，是使用的4个固定的candidate image么？还是根据时间窗口，4个candidate image会相应变动？

from livespeechportraits.

The number of the candidate images set of each video is not fixed. As mentioned above, if your data doesn't include changing camera parameters -- the influence of candidate images set is not obvious. For example, if one video contains 3 different camera parameters, I will select 3 different candidate image set during the training. During testing, you should keep the candidate images fixed.

from livespeechportraits.