This repository is the official implementation of ECM. In this work, we study the weakly supervised temporal action localization and develop the Equivalent Classification Mapping (ECM) mechanism. Paper from arXiv or IEEE.

To install requirements:

conda env create -f environment.yaml

Before running the code, please activate this conda environment.

Download Thumos14 from baiduyun (code:ecmd).

Download ActivityNet1.2 from baiduyun (code:ecmd).

Download ActivityNet1.3 features from baiduyun (code:ecmd).

Please ensure the data structure is as below

├── data
   └── Thumos14
       ├── val
           ├── video_validation_0000051.npz
           ├── video_validation_0000052.npz
           └── ...
       └── test
           ├── video_test_0000004.npz
           ├── video_test_0000006.npz
           └── ...
   └── ActivityNet1.2
       ├── training
           ├── v___dXUJsj3yo.npz
           ├── v___wPHayoMgw.npz
           └── ...
       └── validation
           ├── v__3I4nm2zF5Y.npz
           ├── v__8KsVaJLOYI.npz
           └── ...
   └── ActivityNet1.3
       ├── training
           ├── v___c8enCfzqw.npz
           ├── v___dXUJsj3yo.npz
           └── ...
       └── validation
           ├── v__1vYKA7mNLI.npz
           ├── v__3I4nm2zF5Y.npz
           └── ...
     

To train the ECM model on THUMOS14 dataset, please first run this command:

cd ./tools
python train.py -dataset THUMOS14

To train the ECM model on ActivityNet v1.2 dataset, please run this command:

cd ./tools
python train.py -dataset ActivityNet1.2

To train the ECM model on ActivityNet v1.3 dataset, please run this command:

cd ./tools
python train.py -dataset ActivityNet1.3

To evaluate ECM model on Thumos14, run:

python eval.py -dataset THUMOS14 -weight_file ../checkpoints/THUMOS14_best.pth

To evaluate ECM model on ActivityNet1.2, run:

python eval.py -dataset ActivityNet1.2 -weight_file ../checkpoints/ActivityNet1.2_best.pth

To evaluate ECM model on ActivityNet1.3, run:

python eval.py -dataset ActivityNet1.3 -weight_file ../checkpoints/ActivityNet1.3_best.pth

You can download pre-trained models here:

• THUMOS14_best.pth baiduyun (code:ecmd) trained on THUMOS14 using parameters same as "./experiments/THUMOS14.yaml".
• ActivityNet1.2_best.pth baiduyun (code:ecmd) trained on ActivityNet v1.2 using parameters same as "./experiments/ActivityNet1.2.yaml".
• ActivityNet1.3_best.pth baiduyun (code:ecmd) trained on ActivityNet v1.2 using parameters same as "./experiments/ActivityNet1.3.yaml".

Our model achieves the following performance on :

@article{zhao2022equivalent,
  title={Equivalent classification mapping for weakly supervised temporal action localization},
  author={Zhao, Tao and Han, Junwei and Yang, Le and Zhang, Dingwen},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2022},
  publisher={IEEE}
}