The addon drives a 3D cartoon character by transferring SMPL's pose and global translation into the skeleton of the 3D character. Poses and global translation can be obtained from RGB images using ROMP or any SMPL-based 3D pose estimation model. If the estimation model outputs pose and global position at a high speed, then you can achieve the effect of driving 3D characters in real time in Blender.

Steps:

1. Modify the path of results.npz in server.py and start server.py in the command line.
2. Open Beta.blend or SMPL.blend and click on the triangle in the upper right corner.If a message is displayed in the lower left corner, the addon is running successfully.
3. Go back to the Layout view and click on the small icon in the upper right to get the Texture.
4. Press Ctrl+E to run the addon. At this time, the keyframe that is being transferred is displayed in the command line running server.py.
5. Press the space in Blender to view the character animation.

If you want to use your own video, you need to use ROMP's video.sh to get a npz file.

Steps:

1. Connect the camera and run ROMP's webcam_blender.sh.
2. The rest of the steps are the same as steps 2, 3 of From Video to Animation, where the 3D characters in Blender are driven in real time by pressing Ctrl+E.Press A to stop it.

If you are familiar with Blender and you want to use your own 3D characters, make sure that the skeleton is exactly the same as the SMPL's skeleton and that the bones are named the same as the bones of my 3D characters (of course, you can also change the bone names in the addon). All the 24 bones in SMPL need to be named the same. There is no need to change the name of the finger bones.

If you need a video background in the demo, select Compositing in the top menu bar, click Open Clip in the Movie Clip, and select your video. It is best not to use video as a background when driving in real time to prevent animations from getting too stuck.

• 3D character is downloaded from Mixamo
• Blender 2.8
• Blender Manual
• Blender Python API
• neuron_mocap_live-blender
• QuickMocap-BlenderAddon