Comments (1)
At first, all three models: Audio2Feature, Audio2Headpose, and Feature2Face should be re-trained for any new data.
Other Questions:
1 2d landmarks: 73 points landmarks detector is not open-source, it is a closed-source tool developed by the company. Common used landmarks detectors like face_alignment (68 points) is also work but you need to change the landmarks' semantic settings.
2.1 Yes.
2.2 Actually I didn't do any normalized on tracked 3d points. Sry for that the name of the file 'tracked3D_normalized_pts_tracked3D_normalized_pts_fix_contour.npy' may mislead your way (if you checked the data carefully). The only difference is that I fix the face contour points in the latter file. Contour here means that I fix the contour indices for the reconstruction results. This is about another area about 3D face model tracking. In a word, I fixed the contour indices instead of using sliding contour indices found during the tracking. (If you are familiar with this area, you will understand it quickly).
2.3 3d landmarks are the object coordinates of the face model, i.e., facewarehouse in this project. There is no relationship to the image or something else, it is just the tracked face coordinates of the face model. As illustrated in the paper, I tracked the face on the original resolution. However, our desired results need to cut&crop&resize the image, therefore, 'scale, xc, yc' and other parameters denote the cut&crop&resize parameters. I need these parameters to transform the tracking results& camera parameters to fit the desired results (that is about resolution, face location, etc.)
2.4 'tracked2D_normalized_pts_fix_contour.npy' contains the transformed tracked 2d facial landmarks of the training data. Also sorry for the 'normalized' may mislead your way. Here, using 'transformed' is a better choice. It lies on the image coordinates (512x512).
About the training, please check issue #19. Here I don't illustrate it again. Hope for your understanding.
from livespeechportraits.
Related Issues (20)
- What is the meaning of implementing by C++? HOT 1
- 候选照片,一共四张,是基于什么逻辑进行选择的?
- how can i use it in real time? HOT 1
- Does anyone implement the training code of this project? HOT 1
- How to run demo in "Real-time" HOT 1
- 模型得到的矩阵值可以和ARkit进行映射吗?
- RuntimeError: Found no NVIDIA driver on your system.
- Great project, where does the author achieve real-time performance? HOT 2
- 如何生成自己的模型。从哪里导入我的视频素材生成我自己的模型。
- How to train these models in custom dataset? Any documentation? HOT 1
- What tool did you use to create a sketch from a face image, in case i want to train the image to image transition model?
- 73 facial landmarks HOT 1
- FileNotFoundError: [Errno 2] No such file or directory: './data/May\\mean_pts3d.npy' HOT 1
- 数字人技术交流群请联系VX:metahuman668
- GMMLogLoss for training audio2headpose
- training data download
- Is the Released Models Trained on Whole Video Clip?
- code for data processing, training HOT 2
- REAL TIME 哪里去了?不是说好可以根据音频流来实时输出吗? HOT 2
- Lip sync result HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from livespeechportraits.