Data acquisition and processing about audio2photoreal HOT 3 CLOSED

Hi! Thank you for your interest. The data acquisition directly follows from this repository. The technical report for that codebase provides details on number of cameras, positioning of cameras, etc. For the body pose, we used a similar setup to collect the dataset.

For the raw audio, we fetched this from the dome. Unfortunately I'm not sure I know much more about the details about the actual audio capture. From that .mp4 style files, I just dumped them directly into .npy files. So no actual audio processing happened there.

Lastly, the data processing step requires the full multiview to do the reconstruction. To get a 3D avatar, we render from many sides and do reconstruction across different viewpoints.

Hope this helps!

from audio2photoreal.

Thank you very much for your response!
I still have something that needs to be clarified:

1. The body poses and face expression extraction step needs the full multiview recorded video, right?
2. When do you plan to release the video from the dataset and processing code?

from audio2photoreal.

Regarding 1. yep, both the extraction for face and body needs the full multiview recorded video, which unfortunately we will not be releasing at the moment, and 2. sadly that is still up in the air. At the moment, we are just releasing the photoreal renderings, and 3d pose estimates provided in this repo. If anything changes, I will update the repo to reflect this.

from audio2photoreal.