vchoutas / smplify-x Goto Github PK
View Code? Open in Web Editor NEWExpressive Body Capture: 3D Hands, Face, and Body from a Single Image
Home Page: https://smpl-x.is.tue.mpg.de/
License: Other
Expressive Body Capture: 3D Hands, Face, and Body from a Single Image
Home Page: https://smpl-x.is.tue.mpg.de/
License: Other
Thank you for the great work!
I am new to this line of research. I am amazed by this work. Thank you.
When the script finished, I noticed that the output folder contains 3 subfolders: images, meshes, and results. In the "results" subfolder, I can see there are subfolders for individual frames, each frame subfolder contains pkl files. The number of pkl files is the same as the number of persons indicated by the input keypoint json files. I assume the pkl files are the information about each person in each frame.
My question is how to make sense of the pkl files. I know they are dict python objects. I used a simple python script to read the print the information in the dict python objects. But I don't understand the information. Here is a same print out from one of the pkl files.
camera_rotation:
[[[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]]
camera_translation:
[[-0.93072885 -0.32276985 28.068737 ]]
betas:
[[ 2.2748034 1.6003036 -2.7321897 0.956506 0.43125007 0.7276902
-0.35422274 0.01619153 -0.51857615 0.2500089 ]]
global_orient:
[[ 2.553208 -1.368408 0.497132]]
left_hand_pose:
[[-0.00887376 -0.02925843 0.11824869 0.03584188 -0.00678155 -0.01434492
-0.00700561 0.01624358 -0.01806656 -0.01179078 0.05619476 -0.0107846 ]]
right_hand_pose:
[[ 0.01390133 -0.04411352 0.02681856 -0.08472307 -0.03366644 -0.00826263
0.00735318 0.01106616 0.00353818 -0.00478872 0.01470109 -0.00560154]]
jaw_pose:
[[ 0.43463567 0.00300382 -0.00167876]]
leye_pose:
[[ 0.33339098 0.72704697 -0.10674094]]
reye_pose:
[[ 0.38420925 -1.7449956 0.49268994]]
expression:
[[-0.1779122 -0.05016612 0.09350009 -0.06459138 -0.01217079 -0.00341537
0.01744352 -0.00746865 0.01760298 -0.0005899 ]]
body_pose:
[[ 0.3203223 -0.737228 -0.6891902 -0.7276636 1.7008942 0.28089714
1.2244431 0.19209296 2.3767915 -0.6681881 -0.60998005 -1.2172983
-0.6431202 -0.01528043 0.93774366 0.30256948 0.12850021 0.40082023
-1.5943441 0.73523766 0.905152 0.2594106 -1.4429822 -0.50096047
-0.37165028 -0.15759958 -0.53579974 0.07295594 -0.6267392 -0.52626413
-1.2326773 -1.2687916 ]]
My use scenario is that I have videos with multiple persons (2-3). In addition to the constructed 3D models of the persons, I need to know the relative positions of the persons (and their relative positions to the camera). The obj meshes seem to be centered to each person. So I dug into the pkl files to see if I can get the information I need.
Any pointers to documents or webpages to make sense of output/results would help. Thanks.
Hi,
Very cool work! If a person wanted to try this on their own imageset how should we generate the required keypoints?
Best wishes,
Michael
hi,I have a problem. The code work fine,but output has noting result.
Thank you for your great work. When it is run with fit_smpl.yaml configuration, "SMPL_NEUTRAL.pkl doesn't exist error" happens.
However, only female and male models can be obtained from here.
Is that a problem of code or missing of data?
thanks,
Good morning, this model seems to me extremely exciting!
I have configuration problems, in fact I have this error and I do not understand, keypoints should be generated by myself or should I do something?
This is the error:
python smplifyx/main.py --config cfg_files/fit_smplx.yaml --data_folder DATA_FOLDER --output_folder OUTPUT_FOLDER --visualize="True" --model_folder MODEL_FOLDER --vposer_ckpt VPOSER_FOLDER --part_segm_fn smplx_parts_segm.pkl
Traceback (most recent call last):
File "smplifyx/main.py", line 272, in
main(args)
File "smplifyx/main.py", line 201, in main
for idx, data in enumerate(dataset_obj):
File "/Users/piero******/anaconda3/envs/AVATAR/smplify-x/smplifyx/data_parser.py", line 206, in next
return self.next()
File "/Users/piero********/anaconda3/envs/AVATAR/smplify-x/smplifyx/data_parser.py", line 215, in next
return self.read_item(img_path)
File "/Users/piero********/anaconda3/envs/AVATAR/smplify-x/smplifyx/data_parser.py", line 185, in read_item
use_face_contour=self.use_face_contour)
File "/Users/piero********/anaconda3/envs/AVATAR/smplify-x/smplifyx/data_parser.py", line 54, in read_keypoints
with open(keypoint_fn) as keypoint_file:
FileNotFoundError: [Errno 2] No such file or directory: 'DATA_FOLDER/keypoints/76_img_keypoints.json'
I used to render model with openDR, but the rendered model in your paper can even show the topology of the mesh. I just wonder how to get a rendered model like that? Thanks~
I am using python3.6 , torch 1.1.0 and when i am running the smplifyx/main.py ,
as
python3 smplifyx/main.py --config cfg_files/fit_smplx.yaml --data_folder input/ --output_folder output/ --visualize="True" --model_folder ../models/ --vposer_ckpt ../vposer/ --part_segm_fn ../smplx_parts_segm.pkl
i am getting error,
pyglet.gl.ContextException: Could not create GL context
and the code is getting stuck like this.
Any help?
CUDA_VISIBLE_DEVICES=3,4 python3 smplifyx/main.py --config cfg_files/fit_smplx.yaml --data_folder data --output_folder output --visualize="False" --model_folder model --vposer_ckpt vposer --part_segm_fn smplx_parts_segm.pkl
Traceback (most recent call last):
File "smplifyx/main.py", line 272, in
main(**args)
File "smplifyx/main.py", line 178, in main
camera = camera.to(device=device)
File "/home/wangqy/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 386, in to
return self._apply(convert)
File "/home/wangqy/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 199, in _apply
param.data = fn(param.data)
File "/home/wangqy/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 384, in convert
return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
RuntimeError: CUDA error: out of memory
Hello, My name is Long.
I use the pics from your paper .the left is not the same as the paper and the right is same.two arms backward. can you tell me how do i set the param?
I use the param from the fit_smplx.yaml.
thank you.
this is another question:
No matter I use your .json or mine in the same pic, the result is same as the left effect. so when I change focus_length=500, the effect seems right ,but the shape is not. Curiously, when I use the pic on the bottom and focus_length=5000 , the effect is right including shape.
what's different with two pics? I think the focus_length is same on two pics.but reality is not.
thank you .
Hello, I was trying to understand the code for the model fitting. I have a doubt: in the file fit_single_frame.py
, line 417, there is written:
curr_weights['bending_prior_weight'] = (3.17 * curr_weights['body_pose_weight'])
Why 3.17? What does that number signify?
Thank you
hi:
Thanks for your working . I got an error shown in the picture and there is no window opening during optimization .The code run successfully after setting visualize=False and save_meshes=True, but I still want to kown how to run with visualize=True.
By the way ,can torch-mesh-isect only run in ubuntu18.04? I 'v tried #3 but still not work, and my os is ubuntu16.04.
Thanky ou, for all the help
OS: High Sierra
Device: Mac Book Pro 2014
No GPU
I ran the smplify-x fit on the key points obtained from openpose for 1 image from the smplify-x evaluation dataset. I seem to run into an error with the rendering:
ValueError: Unable to initialize an OpenGL 3+ context
the command I am running is:
python smplifyx/render_results.py --mesh_fns output/meshes
In output/meshes
, i have an obj file within a folder 01_2Djnt/01_2Djnt.onj
. Any recommendations on how to handle the opengl context problem on macOS?
I got the whole code to run and obj was generated, but with Self Intersection set to False.
I faced many issues in setting up the PyTorch Mesh package for the versions of other s/w running.
My question is, since am not using Self Intersection, what exactly am i missing out on? How critical is it? otherwise, I have to recreate the whole setup again from scratch.
Hi, I have this problem that I can not solve the file keypoints was created with OpenPose and is present in the directory, I do not understand the reason for this error?
SMPLIFY-X) bash-3.2$ python smplifyx/main.py --config cfg_files/fit_smplx.yaml --data_folder DATA_FOLDER --output_folder OUTPUT_FOLDER --visualize="True" --model_folder models --vposer_ckpt models/vposer_v1_0
Traceback (most recent call last):
File "smplifyx/main.py", line 272, in <module>
main(**args)
File "smplifyx/main.py", line 201, in main
for idx, data in enumerate(dataset_obj):
File "/Users/piero/Desktop/SMPLIFY-X/smplify-x/smplifyx/data_parser.py", line 206, in __next__
return self.next()
File "/Users/piero/Desktop/SMPLIFY-X/smplify-x/smplifyx/data_parser.py", line 215, in next
return self.read_item(img_path)
File "/Users/piero/Desktop/SMPLIFY-X/smplify-x/smplifyx/data_parser.py", line 185, in read_item
use_face_contour=self.use_face_contour)
File "/Users/piero/Desktop/SMPLIFY-X/smplify-x/smplifyx/data_parser.py", line 54, in read_keypoints
with open(keypoint_fn) as keypoint_file:
FileNotFoundError: [Errno 2] No such file or directory: 'DATA_FOLDER/keypoints/COCO_val2014_000000000459_rendered_keypoints.json'
I've successfully generated several meshes that look good and match the pose. I am trying to blendshape between them in a 3D software (e.g. Modo), but getting bad results because the point order is seemingly different.
All output meshes I'm testing were generated using the neutral model.
Should I expect the same order between output meshes? Am I missing a step somewhere?
helllo thanks for you codes!
where can i find the images and keypoints
can you upload you datasets?
Thank you for making this possible. This is great work!
When I ran the code, I got a Segmentation fault (core dumped) error during "Orientation". Do you have any ideas on how I can fix it? Thanks.
(I am using a machine with Ubuntu 18.04, CUDA 10).
The output:
Processing: ../EHF/images/02_img.jpg
Found Trained Model: ../vposer_v1_0/snapshots/TR00_E096.pt
Camera initialization done after 1.3077
Camera initialization final loss 8734.7334
Stage 000 done after 4.3601 seconds
Stage 001 done after 1.4332 seconds
Stage 002 done after 1.0444 seconds
Orientation: 0%| | 0/1 [00:06<?, ?it/sSegmentation fault (core dumped)###8 | 3/5 [00:06<00:05, 2.75s/it]
An update:
I ran the command again with gdb. The following is the last few lines of the output:
Stage 000 done after 4.3009 seconds
Stage 001 done after 1.4249 seconds
Stage 002 done after 1.0407 seconds
Orientation: 0%| | 0/1 [00:06<?, ?it/s]
Thread 1 "python3" received signal SIGSEGV, Segmentation fault.00:05, 2.72s/it]
0x00007fffa5dc0f4d in ?? ()
from /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/lib/libcudart-f7fdd8d7.so.9.0
Another update:
Since from the segmentation fault came from libcudart-f7fdd8d7.so.9.0, I suspected that the PyTorch is with a CUDA 9.0 library. Therefore, I updated the PyTorch (with CUDA 10). My machine supports CUDA (I checked it by running command nvcc -V). After the update, I ran the code again. I got a different error at a similar point. The output (from gdb):
Stage 000 done after 3.9944 seconds
Stage 001 done after 0.7773 seconds
Stage 002 done after 3.9981 seconds
Orientation: 0%| | 0/1 [00:08<?, ?it/sCuda failure src/bvh_cuda_op.cu:877: 'an illegal memory access was encountered't]
[Thread 0x7fff7ffff700 (LWP 5409) exited]
[Thread 0x7fff92966700 (LWP 5407) exited]
[Thread 0x7ffff2a96700 (LWP 5406) exited]
[Thread 0x7ffff0295700 (LWP 5405) exited]
[Thread 0x7fffefa94700 (LWP 5404) exited]
[Thread 0x7ffff7fdd700 (LWP 5395) exited]
[Inferior 1 (process 5395) exited normally]
Does anyone have any ideas what is wrong? Thx.
Can someone tell where to get this PKL
Hello
Do you plan on releasing the training codes for the project?
thank you for your great work!!!
it's really help.
smplify-x/smplifyx/cmd_parser.py
Line 135 in 6c3dcbe
Lines 120 to 130 in 2de38f3
I would like to ask if this project can only control the model of the hand without changing the body.
thx.
When I run the command with smplx model
python smplifyx/main.py --config cfg_files/fit_smplx.yaml
--data_folder DATA_FOLDER
--output_folder OUTPUT_FOLDER
--visualize="True/False"
--model_folder MODEL_FOLDER
--vposer_ckpt VPOSER_FOLDER
--part_segm_fn smplx_parts_segm.pkl
I have follow error:
File "smplifyx/main.py", line 273, in
main(**args)
File "smplifyx/main.py", line 263, in main
**args)
File "D:\Progetti\Tensorflow\GAN\smplify-x\smplifyx\fit_single_frame.py", line 444, in fit_single_frame
use_vposer=use_vposer)
File "D:\Progetti\Tensorflow\GAN\smplify-x\smplifyx\fitting.py", line 175, in run_fitting
loss = optimizer.step(closure)
File "D:\Progetti\Tensorflow\GAN\smplify-x\smplifyx\optimizers\lbfgs_ls.py", line 280, in step
orig_loss = closure()
File "D:\Progetti\Tensorflow\GAN\smplify-x\smplifyx\fitting.py", line 253, in fitting_func
**kwargs)
File "D:\Program Files\Python37_64\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Progetti\Tensorflow\GAN\smplify-x\smplifyx\fitting.py", line 372, in forward
if self.use_joints_conf else
RuntimeError: The size of tensor a (118) must match the size of tensor b (25) at non-singleton dimension 1
with SMP it works.
What should be the issue?
Hi,
Just for the record, some days ago I found this paper and find it very amazing. I spent some time playing with the code and making some changes to better understand it.
To let them open to the public, I published them here:
https://github.com/ortegatron/playing_smplifyx/
It allows to run on a single image, view the progress made by the optimizer and use OpenPose on the fly.
Maybe someone finds it usefull.
Regards and thanks for publishing this!
Thanks a lot for releasing this great job!
SMPLX is the combination of SMPLH and FLAME. Would you please provide the vertex index map between SMPLH/FLAME and SMPLX? Thanks a lot.
@vchoutas Hi it's Lotayou again, I disabled mesh intersection part and finally the fitting script runs okay. However it takes almost 1 min to fit a single image, as shown below:
I'm wondering if this is the normal speed? For the record, as the script was running I checked the nvidia-smi
and confirmed that the code was running on GPU. Is there anyway to speedup? Thanks!
when i run
python35.exe smplifyx/main.py --config "cfg_files/fit_smpl.yaml" --data_folder "D:\\Download\\out_data\\test " --output_folder "D:\\Download\\out_data\\test_out_txt"
Tips:
`
Traceback (most recent call last):
File "smplifyx/main.py", line 272, in
main(**args)
File "smplifyx/main.py", line 117, in main
male_model = smplx.create(gender='male', **model_params)
File "E:\ProgramData\python35\lib\site-packages\smplx-0.1.11-py3.5.egg\smplx\body_models.py", line 97, in create
File "E:\ProgramData\python35\lib\site-packages\smplx-0.1.11-py3.5.egg\smplx\body_models.py", line 185, in init
AssertionError: Path models does not exist!
`
so ,i want know, models where? thk
Hi,
I was able to run the smplify-x.
step 02
I generate the keypoints from openpose and generate the json file as follows.(plz remove the .pdf and add .json )
tummy_man_keypoints.pdf
step 03
Then i provide the step 01 image and the step 02 tummy_man_keypoints.json to the smplify-x.
i run the smplifyx,
python3 smplifyx/main.py --config cfg_files/fit_smplx.yaml
--data_folder DATA_FOLDER
--output_folder OUTPUT_FOLDER
--visualize="False"
--model_folder MODEL_FOLDER
--vposer_ckpt VPOSER_FOLDER
inside DATA_FOLDER contain two subfolders, images, where the images are located, and keypoints, where the OpenPose output should be stored.
inside MODEL_FOLDER put the SMPL-X Model
inside VPOSER_FOLDER put the VPoser: Variational Human Pose Prior
OUTPUT OF THE CONSOLE
Processing: /home/xxxx/simplifyx/smplify-x/DATA_FOLDER/images/tummy_man.png
/home/xxxx/simplifyx/smplify-x/OUTPUT_FOLDER/meshes/tummy_man
/home/xxxx/simplifyx/smplify-x/OUTPUT_FOLDER/results/tummy_man
Found Trained Model: /home/xxxx/simplifyx/smplify-x/VPOSER_FOLDER/snapshots/TR00_E096.pt
Camera initialization done after 2.4678
Camera initialization final loss 4296.7549
Stage 000 done after 5.3883 seconds
Stage 001 done after 3.4187 seconds
Stage 002 done after 11.2370 seconds
Stage 003 done after 9.5738 seconds
Stage 004 done after 8.8275 seconds
Body fitting Orientation 0 done after 38.4547 seconds
Body final loss val = 9400.93066
Orientation: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:38<00:00, 38.46s/it]
Processing the data took: 00 hours, 00 minutes, 46 seconds
Then obj file was generated.The screenshot of the view is as follows.
Seems to be the output is wrong. Can any one help me to get the correct output?
I have run your code to generate a mesh, but when i try to overlay the saved mesh to the image. The color image didn't change.How should I set the pyrender paramters to get the right result.Thank you in advance~
This is the input image with 25 annotated key points:
This is the output I obtained by fitting the SMPL model :
I used the default setting and my command line input is:
--config "../cfg_files/fit_smpl.yaml" --data_folder $IMAGE_PATH --output_folder "./output" --visualize="True" --model_folder $MODEL_PATH --vposer_ckpt $VPOSE --part_segm_fn smplx_parts_segm.pkl
Appreciate it if you know that's going wrong. Thank you.
With GTX1080Ti GPU and right env setup, I tried several times to run the demo but failed.
In case I misuse the wrong input image ,I use the 1st image appear in the paper, but I still get the same error:
(cp36) sam@sam-HP-Z620-Workstation:/home/sdb/ab/MPI_SMPL/smplify-x$ python smplifyx/main.py --config cfg_files/fit_smplx.yaml --data_folder input --output_folder output --visualize="False" --model_folder models --vposer_ckpt vposer_v1_0 --part_segm_fn smplx_parts_segm.pkl --use_cuda=False --interpenetration=False
Processing: input/images/1.png
Found Trained Model: vposer_v1_0/snapshots/TR00_E096.pt
Traceback (most recent call last):
File "smplifyx/main.py", line 272, in
main(**args)
File "smplifyx/main.py", line 262, in main
**args)
File "/home/sdb/ab/MPI_SMPL/smplify-x/smplifyx/fit_single_frame.py", line 274, in fit_single_frame
focal_length=focal_length, dtype=dtype)
File "/home/sam/miniconda3/envs/cp36/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 49, in decorate_no_grad
return func(*args, **kwargs)
File "/home/sdb/ab/MPI_SMPL/smplify-x/smplifyx/fitting.py", line 75, in guess_init
pose_embedding, output_type='aa').view(1, -1) if use_vposer else None
File "vposer_v1_0/vposer_smpl.py", line 114, in decode
if output_type == 'aa': return VPoser.matrot2aa(Xout)
File "vposer_v1_0/vposer_smpl.py", line 152, in matrot2aa
pose = tgm.rotation_matrix_to_angle_axis(homogen_matrot).view(batch_size, 1, -1, 3).contiguous()
File "/home/sam/miniconda3/envs/cp36/lib/python3.6/site-packages/torchgeometry/core/conversions.py", line 233, in rotation_matrix_to_angle_axis
quaternion = rotation_matrix_to_quaternion(rotation_matrix)
File "/home/sam/miniconda3/envs/cp36/lib/python3.6/site-packages/torchgeometry/core/conversions.py", line 302, in rotation_matrix_to_quaternion
mask_c1 = mask_d2 * (1 - mask_d0_d1)
File "/home/sam/miniconda3/envs/cp36/lib/python3.6/site-packages/torch/tensor.py", line 325, in rsub
return _C._VariableFunctions.rsub(self, other)
RuntimeError: Subtraction, the -
operator, with a bool tensor is not supported. If you are trying to invert a mask, use the ~
or bitwise_not()
operator instead.
It seems some internal calculation problem, but I do not have time to check the code details. Can someone help me?
Thanks.
// Sorry I haven't got the redistribution permission of the image in the paper, it has to be removed.
So I only put the keypoints result here.
OpenPose keypoint json file content:
{"version":1.3,"people":[{"person_id":[-1],"pose_keypoints_2d":[97.9395,59.9321,0.94477,94.6404,121.035,0.904376,48.3801,121.08,0.883265,48.3408,200.294,0.802151,21.9325,269.598,0.842027,139.198,119.412,0.844284,139.249,213.469,0.923468,111.101,160.707,0.907425,73.1316,277.938,0.723224,45.0834,276.284,0.706072,64.8885,403.368,0.796988,81.3546,522.287,0.655121,99.6243,279.561,0.707398,94.5705,398.47,0.805579,112.759,523.927,0.633479,86.4163,48.352,0.959138,109.468,48.3612,0.94057,71.5461,53.3043,0.952888,124.298,53.3778,0.936655,71.5543,545.399,0.102387,79.7951,542.131,0.158925,124.321,537.148,0.508061,33.5219,550.336,0.620125,35.1681,543.788,0.562077,88.0524,538.827,0.25901],"face_keypoints_2d":[72.6313,47.1096,0.759064,73.4384,53.8361,0.793032,75.3219,60.2935,0.799174,76.9362,67.02,0.799732,78.8196,73.7465,0.813515,80.9721,80.2039,0.865062,84.2008,86.1232,0.826529,89.3129,90.6972,0.846112,95.5013,92.5806,0.856205,102.497,90.9662,0.858706,107.609,86.6613,0.850709,112.183,80.742,0.835608,115.412,74.5536,0.835211,117.564,67.5581,0.872575,118.909,60.5626,0.846016,120.524,53.567,0.808359,121.062,46.5715,0.712315,77.4743,45.2262,0.836439,80.9721,43.0737,0.908461,85.008,42.2666,0.90444,89.3129,43.0737,0.925983,93.0797,44.419,0.837476,101.421,44.9572,0.852778,105.456,43.0737,0.900674,109.761,42.2666,0.861295,114.066,43.0737,0.856579,117.564,45.4953,0.85144,97.3847,49.2621,0.8834,97.6537,53.8361,0.9431,97.3847,58.6792,0.939789,97.6537,63.2532,0.902095,92.8107,66.4819,0.850246,94.9632,67.02,0.910726,97.3847,67.8272,0.894393,99.8062,67.289,0.909263,101.69,66.7509,0.895275,82.0483,48.993,0.910519,85.008,47.3787,0.934796,88.5057,47.3787,0.895105,91.4654,49.8002,0.882799,88.2367,50.8765,0.871825,84.7389,50.8765,0.924408,103.304,50.0693,0.892791,106.533,47.6477,0.907411,110.03,47.3787,0.921187,112.99,49.2621,0.940145,110.3,50.8765,0.908453,106.802,50.8765,0.92475,87.6986,73.4774,0.928661,91.1963,71.863,0.882647,94.425,71.3249,0.925195,97.1156,71.863,0.918953,99.8062,71.3249,0.925565,102.766,72.1321,0.964395,105.456,73.7465,0.89528,103.035,79.1276,0.883822,99.5372,81.8182,0.904576,96.3084,82.3563,0.984643,93.6179,81.8182,0.940493,89.851,78.8586,0.91854,89.0439,73.7465,0.95698,94.425,73.4774,0.896883,97.1156,73.4774,0.898466,99.8062,73.4774,0.893764,104.38,74.2846,0.935939,99.5372,78.0514,0.902504,96.5775,78.5895,0.897735,93.8869,78.0514,0.961007,86.6223,48.4549,0.811237,108.147,48.4549,0.846577],"hand_left_keypoints_2d":[103.882,156.753,0.540482,96.9814,150.197,0.242396,84.5597,139.156,0.31344,78.694,133.98,0.645887,76.6237,132.6,0.593826,101.122,128.804,0.466691,101.467,119.143,0.862446,102.502,111.897,0.864072,104.572,105.341,0.856388,106.988,131.564,0.760933,95.6012,129.494,0.845451,87.3201,132.6,0.770535,83.1796,137.775,0.730734,110.093,136.74,0.862037,97.3265,137.775,0.950933,90.4255,142.261,0.686533,85.2498,146.056,0.620184,111.818,142.261,0.794224,102.847,143.296,0.738635,96.9814,145.021,0.708584,94.2211,146.402,0.715982],"hand_right_keypoints_2d":[26.2168,275.841,0.0558444,25.3099,267.982,0.175173,23.7984,265.563,0.300864,24.1007,268.284,0.53134,22.8915,277.958,0.377694,13.5201,283.097,0.850246,22.5892,286.724,0.679191,28.9375,288.538,0.801663,35.5882,291.259,0.797258,12.9155,288.84,0.742155,22.8915,293.073,0.668555,31.0536,294.282,0.747965,36.4951,295.793,0.518082,14.7293,292.77,0.584971,22.5892,297.91,0.607702,29.8444,298.816,0.578564,36.7974,298.212,0.579198,18.0546,295.189,0.430246,22.5892,300.63,0.741699,28.3329,302.444,0.808895,33.7744,303.956,0.835708],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]}]}
need some help on how to set the value for focal length in the yaml file. the setting there is 5000 by default.
i am using an Oneplus camera and the sony imx sensor is rated to have a 25mm focal length while the aperture is f/1.7.
Now how to adjust the values in yaml for my specific camera?
Hi,
Is there a way to get the SMPL parameters?
I receive a mesh and a pkl file only.
Thanks in advance,
Alon
Hi, I just wanted to note this is amazing work!
I've been trying to test Smplify-X on a custom image of a person standing up.
I've been getting this error:
Traceback (most recent call last): File "smplifyx/render_results.py", line 99, in <module> mesh_fn = key_handler.get_mesh_fn() File "smplifyx/render_results.py", line 54, in get_mesh_fn return self.mesh_fns[self.idx] IndexError: list index out of range
Any idea how to fix this?
Thanks!
When I run this code in a folder with many some images, the code will stop sometimes. The error is
Found Trained Model: vposer_v1_0/snapshots/TR00_E096.pt
NaN loss value, stopping!
Camera initialization done after 2.0577
Traceback (most recent call last):
File "smplifyx/main.py", line 276, in <module>
main(**args)
File "smplifyx/main.py", line 266, in main
**args)
File "/home/smplify-x/smplifyx/fit_single_frame.py", line 393, in fit_single_frame
cam_init_loss_val))
TypeError: unsupported format string passed to NoneType.__format__
I think this error is about camera initialization loss. How to solve this problem? The second time I run this code for the same error image, it runs well.
If we set gender_gt in openpose keypoints file. Run smplifyx/main will occur an error:
Traceback (most recent call last):
File "smplifyx/main.py", line 272, in <module>
main(**args)
File "smplifyx/main.py", line 236, in main
if gender == 'neutral':
UnboundLocalError: local variable 'gender' referenced before assignment
Is this a typing mistake?
if 'gender_gt' in person_data:
gender_pd.append(person_data['gender_gt'])
smplify-x/smplifyx/data_parser.py
Line 95 in be588c4
I think it's should use gender_gt.append
, not gender_pd.append
at here.
if 'gender_gt' in person_data:
gender_gt.append(person_data['gender_gt'])
I tested after this change the above UnboundLocalError: local variable 'gender' referenced before assignment
will disappear.
Sorry, but the program does not work without CUDA? I would like to try it on Mac with CPU
Thanks for your code~, Recently, I have carefully read your paper, the paper says annealing scheme is used for the weights γ in the data term EJ (Eq. 5). Is it correspond the yaml's configuration?
The index seems to have something wrong, the total number of hand joints is 42,so the index should from 25 to 67 ? I also have a problem that the paper says there are total 30 joints of the hand, but the number of openpose is 42, I'm really confused about that.
I'm running the model on the EHF evaluation dataset using the command in the README:
python smplifyx/main.py --config cfg_files/fit_smplx.yaml
--data_folder DATA_FOLDER
--output_folder OUTPUT_FOLDER
--model_folder MODEL_FOLDER
--vposer_ckpt VPOSER_FOLDER
--part_segm_fn smplx_parts_segm.pkl
then compositing the original image with the output rendered mesh and often see an offset that seems to be due to a poor camera translation estimate:
In the paper, do you do an additional refinement after the full fitting (using i.e. projected 3D joints) to improve camera translation or have I set up something incorrectly?
First,
use own images(4 of .jpg, 1 of .png) and keypoints with openpose (--write_json)
i run
$ python smplifyx/main.py --config cfg_files/fit_smplx.yaml --data_folder data --output_folder output --visualize ="True" --model_folder model/models --vposer_ckpt vposer --part_segm_fn smplx_parts_segm.pkl
but error occured,
Traceback (most recent call last):
File "smplifyx/main.py", line 272, in
main(**args)
File "smplifyx/main.py", line 262, in main
**args)
File "/data/3d_pose/10_smplify-x/smplifyx/fit_single_frame.py", line 188, in fit_single_frame
vposer, _ = load_vposer(vposer_ckpt, vp_model='snapshot')
File "/usr/local/lib/python3.6/dist-packages/human_body_prior/tools/model_loader.py", line 56, in load_vposer
ps, trained_model_fname = expid2model(expr_dir)
File "/usr/local/lib/python3.6/dist-packages/human_body_prior/tools/model_loader.py", line 36, in expid2model
default_ps_fname = glob.glob(os.path.join(expr_dir,'*.ini'))[0]
IndexError: list index out of range
how can i running right? :(
and i don't have '*.ini' file .. downloads ehf, homogenus,vposer,smplx but not there
#33
So, after the above issue, the code is running but no output meshes are being created.
The code is stuck again.
The output folder has different sub-folders and only a single pkl file.
Stage 000 done after 2.5709 seconds
Orientation: 0%| | 0/1 [00:02<?, ?it/s████████████████▏ | 1/5 [00:02<00:10, 2.57s/it]
Stage 001 done after 1.6226 seconds
Orientation: 0%| | 0/1 [00:04<?, ?it/s███████████████████████████████████████████▍ | 2/5 [00:04<00:06, 2.29s/it]
Stage 002 done after 3.1877 seconds
Orientation: 0%| | 0/1 [00:07<?, ?it/s█████████████████████
█████████████████████████████████████████████████▌ | 3/5 [00:07<00:05, 2.56s/it]
Cuda failure src/bvh_cuda_op.cu:751: 'an illegal memory access was encountered'
what's wrong about this?
Where can this file be found?
Not sure what I did wrong ... installed all dependencies and optional dependencies but get this error when running on an image. Can anyone help please :) Thanks!
Processing: ./data/images/im0117.jpg
Found Trained Model: ./vposer_v1_0/snapshots/TR00_E096.pt
Camera initialization done after 1.1841
Camera initialization final loss 1358.9475
Orientation: 0%| | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last): | 0/5 [00:00<?, ?it/s]
File "./smplify-x/smplifyx/main.py", line 272, in
main(**args)
File "./smplify-x/smplifyx/main.py", line 262, in main
**args)
File "/home/mby/smpl_test/smplify-x/smplifyx/fit_single_frame.py", line 443, in fit_single_frame
use_vposer=use_vposer)
File "/home/mby/smpl_test/smplify-x/smplifyx/fitting.py", line 167, in run_fitting
loss = optimizer.step(closure)
File "/home/mby/smpl_test/smplify-x/smplifyx/optimizers/lbfgs_ls.py", line 280, in step
orig_loss = closure()
File "/home/mby/smpl_test/smplify-x/smplifyx/fitting.py", line 245, in fitting_func
**kwargs)
File "/home/mby/anaconda3/envs/tf/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/mby/smpl_test/smplify-x/smplifyx/fitting.py", line 364, in forward
if self.use_joints_conf else
RuntimeError: The size of tensor a (118) must match the size of tensor b (25) at non-singleton dimension 1
This is a really great work!!!
Is anybody get an error when running the script, and how to fix it?
(CUDA version:V9.0.176; cudnn version:7.0.5) The error is as follows:
Error inside sort: radix_sort: failed on 2nd step: invalid argument████████████████████▌ | 3/5 [00:37<00:25, 12.84s/it]
Cuda failure src/bvh_cuda_op.cu:877: 'invalid argument'
Hi, I have some questions regarding the pose prior loss line385
# Calculate the loss from the Pose prior
if use_vposer:
pprior_loss = (pose_embedding.pow(2).sum() *
self.body_pose_weight ** 2)
else:
pprior_loss = torch.sum(self.body_pose_prior(
body_model_output.body_pose,
body_model_output.betas)) * self.body_pose_weight ** 2
The body_pose_prior takes input of both body_pose and betas if use_vposer==False. However, the pose prior is trained on the AMASS dataset where the SMPL model is used. Since SMPL and SMPL-X do not share shape parameters, will it cause problems when I use pose prior with the SMPL-X model?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.