This repository is the official implementation of BackTAL. In this work, we convert existing action-click supervision to the background-click supervision and develop a novel method, called BackTAL. Paper from arXiv or IEEE.
To install requirements:
conda env create -f environment.yaml
Download pre-extracted I3D features of Thumos14, ActivityNet1.2 and HACS dataset from BaiduYun with code back.
Please ensure the data structure is as below
├── data
└── Thumos14
├── val
├── video_validation_0000051.npz
├── video_validation_0000052.npz
└── ...
└── test
├── video_test_0000004.npz
├── video_test_0000006.npz
└── ...
└── ActivityNet1.2
├── training
├── v___dXUJsj3yo.npz
├── v___wPHayoMgw.npz
└── ...
└── validation
├── v__3I4nm2zF5Y.npz
├── v__8KsVaJLOYI.npz
└── ...
└── HACS
├── training
├── v_0095rqic1n8.npz
├── v_62VWugDz1MY.npz
└── ...
└── validation
├── v_008gY2B8Pf4.npz
├── v_00BcXeG1gC0.npz
└── ...
The raw annotations of THUMOS14 dataset are under directory './data/THUMOS14/human_anns'.
You can download checkpoints for Thumos14, ActivityNet1.2 and HACS dataset from BaiduYun with code back. These models are trained on Thumos14, ActivityNet1.2 or HACS using the configuration file under the directory "./experiments/". Please put these checkpoints under directory "./checkpoints".
Before running the code, please activate the conda environment.
To evaluate BackTAL model on Thumos14, run:
cd ./tools
python eval.py -dataset THUMOS14 -weight_file ../checkpoints/THUMOS14.pth
To evaluate BackTAL model on ActivityNet1.2, run:
cd ./tools
python eval.py -dataset ActivityNet1.2 -weight_file ../checkpoints/ActivityNet1.2.pth
To evaluate BackTAL model on HACS, run:
cd ./tools
python eval.py -dataset HACS -weight_file ../checkpoints/HACS.pth
Our model achieves the following performance:
| threshold | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 |
|---|---|---|---|---|---|
| mAP | 54.4 | 45.5 | 36.3 | 26.2 | 14.8 |
| threshold | average-mAP | 0.50 | 0.75 | 0.95 |
|---|---|---|---|---|
| mAP | 27.0 | 41.5 | 27.3 | 4.7 |
| threshold | average-mAP | 0.50 | 0.75 | 0.95 |
|---|---|---|---|---|
| mAP | 20.0 | 31.5 | 19.5 | 4.7 |
To train the BackTAL model on THUMOS14 dataset, please run this command:
cd ./tools
python train.py -dataset THUMOS14
To train the BackTAL model on ActivityNet v1.2 dataset, please run this command:
cd ./tools
python train.py -dataset ActivityNet1.2
To train the BackTAL model on HACS dataset, please run this command:
cd ./tools
python train.py -dataset HACS
@article{yang2021background,
title={Background-Click Supervision for Temporal Action Localization},
author={Yang, Le and Han, Junwei and Zhao, Tao and Lin, Tianwei and Zhang, Dingwen and Chen, Jianxin},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
year={2021},
publisher={IEEE}
}
For any discussions, please contact [email protected].
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent