Official pytorch project of ACMMM2022 accepted paper "Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Space".

[Paper & Supp] [Poster PDF] [Poster PPT] [Slides] [Video] [Audio] [Subtitles]

Notes:

• 2022.10.15: Our previous work for deterministic human motion prdiction MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction (ICCV2021) has been released, and another offical reimplementation of this project based on Jittor is also available.
• 2022.11.16: Another good work for stock ranking or selection Learning Multiple Stock Trading Patterns with Temporal Routing Adaptor and Optimal Transport (TRA, Lin et al., KDD2021) which has a similar idea with ours is available at Microsoft QLib. We also privide an unofficial reimplementation.

1. Lingwei Dang, School of Computer Science and Engineering, South China University of Technology, China, [email protected]
2. Yongwei Nie, School of Computer Science and Engineering, South China University of Technology, China, [email protected]
3. Chengjiang Long, Meta Reality Lab, USA, [email protected]
4. Qing Zhang, School of Computer Science and Engineering, Sun Yat-sen University, China, [email protected]
5. Guiqing Li, School of Computer Science and Engineering, South China University of Technology, China, [email protected]

We propose a novel sampling method converts the sampling of the distribution into randomly sampling of points from an auxiliary space for diverse and accurate sampling.

Nvidia RTX 3090
Python                 3.9.7
matplotlib             3.5.0
numpy                  1.20.3
opencv-python          4.5.4.60
pandas                 1.4.2
PyYAML                 6.0
tensorboard            2.7.0
tensorboardX           2.4.1
torch                  1.10.0+cu113
torchvision            0.11.1+cu113
scipy                  1.7.2
scikit-learn           1.0.1

Dataset and pretrained models can be found via the Diverse Sampling Resources Link, download them and then

• unzip dataset.zip to ./dataset
• unzip pretrained.zip to ./ckpt/pretrained
• unzip classifier.zip to ./ckpt/classifier

then the dictionary becomes:

diverse_sampling
├─dataset
│  │  .gitignore
│  │  data_3d_h36m.npz
│  │  data_3d_h36m_test.npz
│  │  data_3d_humaneva15.npz
│  │  data_3d_humaneva15_test.npz
│  │  h36m_valid_angle.p
│  │  humaneva_valid_angle.p
│  ├─data_multi_modal
│  │      data_candi_t_his25_t_pred100_skiprate20.npz
│  │      t_his25_1_thre0.500_t_pred100_thre0.100_filtered_dlow.npz
│  └─humaneva_multi_modal
│          data_candi_t_his15_t_pred60_skiprate15.npz
│          t_his15_1_thre0.500_t_pred60_thre0.010_index_filterd.npz
└─ckpt
   ├─classifier
   │     .gitignore
   │      h36m_classifier.pth
   │      humaneva_classifier.pth
   └─pretrained
           .gitignore
           h36m_t1.pth
           h36m_t2.pth
           humaneva_t1.pth
           humaneva_t2.pth   

• evaluate on Human3.6M:

  python main.py --exp_name=h36m_t2 --is_load=1 --model_path=ckpt/pretrained/h36m_t2.pth

• evaluate on HumanEva-I:

  python main.py --exp_name=humaneva_t2 --is_load=1 --model_path=ckpt/pretrained/humaneva_t2.pth

• For Human3.6M:

  python main_classifier.py --exp_name=h36m_t2

• For HumanEva-I:

  python main_classifier.py --exp_name=humaneva_t2

• train on Human3.6M:

  • train CVAE on Human3.6M:

    python main.py --exp_name=h36m_t1 --is_train=1

  • train DiverseSampling on Human3.6M:

    python main.py --exp_name=h36m_t2 --is_train=1

• train on HumanEva-I:

  • train CVAE on HumanEva-I:

    python main.py --exp_name=humaneva_t1 --is_train=1

  • train DiverseSampling on HumanEva-I:

    python main.py --exp_name=humaneva_t2 --is_train=1

If you use our code, please cite our work

@inproceedings{dang2022diverse,
  title={Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Space},
  author={Dang, Lingwei and Nie, Yongwei and Long, Chengjiang and Zhang, Qing and Li, Guiqing},
  booktitle={Proceedings of the 30th ACM International Conference on Multimedia},
  pages={5162--5171},
  year={2022}
}

We follow the code framework of our previous work MSR-GCN (ICCV 2021), and some code was adapted from DLow by Ye Yuan, and GSPS by Wei Mao.

MIT