This repository contains training and inference code for the following paper:
TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style
Chaitanya Patel*, Zhouyingcheng Liao*, Gerard Pons-Moll
CVPR 2020 (ORAL)
[ArXiv] [Project Website] [Dataset Repo] [Youtube]
old-t-shirt_female | t-shirt_female | t-shirt_male |
---|---|---|
![]() |
![]() |
![]() |
- [02-08-2020] Female and male pant weights added.
- [19-07-2020] Male shirt weights added.
- [12-07-2020] Female shirt weights added.
- [28-06-2020] Female t-shirt weights added.
- [25-06-2020] Minor bug fixes and male t-shirt weights added.
- [17-06-2020] Inference script and female old-t-shirt weights added.
python3
pytorch
scipy
chumpy
psbody.mesh
- Download and prepare SMPL model and TailorNet data from dataset repository.
- Set DATA_DIR and SMPL paths in
global_var.py
file accordingly. - Download trained model weights in a directory and set its path to MODEL_WEIGHTS_PATH variable in
global_var.py
.- old-t-shirt_female_weights (4.1 GB)
- t-shirt_male_weights (2.0 GB)
- t-shirt_female_weights (2.0 GB)
- shirt_female_weights (2.5 GB)
- shirt_male_weights (2.5 GB)
- pant_female_weights and pant_male_weights available here
- [Other garments coming soon]
- Set output path in
run_tailornet.py
and run it to predict garments on some random inputs. You can play with different inputs. You can also run inference on motion sequence data. - To visualize predicted garment using blender, run
python run_tailornet.py render
. (Blender needs to be installed.)
... evaluated using evaluate
function in utils/eval.py
.
garment_class | gender | TailorNet Baseline | TailorNet Mixture Model |
---|---|---|---|
old-t-shirt | female | 11.1 | 10.7 |
t-shirt | female | 12.6 | 12.3 |
t-shirt | male | 11.4 | 11.2 |
shirt | female | 14.2 | 14.1 |
shirt | male | 12.7 | 12.5 |
pant | female | 4.7 | 4.8 |
pant | male | 8.1 | 8.1 |
- Set global variables in
global_var.py
, especially LOG_DIR where training logs will be stored. - Set config variables like gender and garment class in
trainer/base_trainer.py
(or pass them via command line) and runpython trainer/base_trainer.py
to train TailorNet MLP baseline. - Similarly, run
python trainer/lf_trainer.py
to train low frequency predictor andtrainer/ss2g_trainer.py
to train shape-style-to-garment(in canonical pose) model. - Run
python trainer/hf_trainer.py --shape_style <shape1>_<style1> <shape2>_<style2> ...
to train pivot high frequency predictors for pivots<shape1>_<style1>
,<shape2>_<style2>
, and so on. SeeDATA_DIR/<garment_class>_<gender>/pivots.txt
to know available pivots. - Use
models.tailornet_model.TailorNetModel
with appropriate logdir arguments to do prediction.
Cite us if you use our model, code or data:
@inproceedings{patel20tailornet,
title = {TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style},
author = {Patel, Chaitanya and Liao, Zhouyingcheng and Pons-Moll, Gerard},
booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {jun},
organization = {{IEEE}},
year = {2020},
}
- Thanks to Bharat for many fruitful discussions and for
smpl_lib
library taken from his MultiGarmentNet repo's lib folder. - Thanks to Garvita for helping out during the onerous procedure of data generation.
For any doubt or concert about the code, raise an issue on this repository.