This is the official code for the paper "Mutual-Learning Knowledge Distillation for Nighttime UAV Tracking".

Nighttime unmanned aerial vehicle (UAV) tracking has been facilitated with indispensable plug-and-play low-light enhancers. However, the introduction of low-light enhancers increases the extra computational burden for the UAV, significantly hindering the development of real-time UAV applications. Meanwhile, these state-of-the-art (SOTA) enhancers lack tight coupling with the advanced daytime UAV tracking approach. To solve the above issues, this work proposes a novel mutual-learning knowledge distillation framework for nighttime UAV tracking, i.e., MLKD. This framework is constructed to learn a compact and fast nighttime tracker via knowledge transferring from the teacher and knowledge sharing among various students. Specifically, an advanced teacher based on a SOTA enhancer and a superior tracking backbone is adopted for guiding the student based only on the tight coupling-aware tracking backbone to directly extract nighttime object features. To address the biased learning of a single student, diverse lightweight students with different distillation methods are constructed to focus on various aspects of the teacher's knowledge. Moreover, an innovative mutual-learning room is designed to elect the superior student candidate to assist the remaining students frame-by-frame in the training phase. Furthermore, the final best student, i.e., MLKD-Track, is selected through the testing dataset. Extensive experiments demonstrate the effectiveness and superiority of MLKD and MLKD-Track. The practicality of the MLKD-Track is verified in real-world tests with different challenging situations.

python 3.8.3
pytorch 1.11.0

Before training, you need to prepare the training set of the NAT2021-test and DarkTrack2021 dataset. Also, the teacher model is based on the SOTA enhancer DarkLighter and the advanced tracker SiamRPN++(), the teacher model parameters can be gotten from HERE(code:team).

Run train_kd.py.

CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 ./tools/train_kd.py --cfg experiments/siamrpn_alex_dwxcorr_16gpu/config.yaml 

Run train_ml.py.

CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 ./tools/train_ml.py --cfg experiments/siamrpn_alex_dwxcorr_16gpu/config.yaml 

Download our pretrained model MLKD-Track(code:mlkd).

Take the test of MLKD-Track as an example:

CUDA_VISIBLE_DEVICES="0" python tools/test.py --snapshot='/home/user/V4R/LYF/pysot-master2/snapshot/mlkd.pth'

If you want to evaluate the trackers mentioned above, please put those results into results directory as results/<dataset_name>/<tracker_name>.

python tools/eval.py                              

If you have any questions, please contact me.

Yufeng Liu

Email: [email protected].

• The code is implemented based on pysot. We would like to express our sincere thanks to the contributors.
• We would also like to thank SiamRPN++ tracker and enhancer for their efforts.