Implementation of brand new video augmentation strategy for video action recognition with 3D CNN.

• reference
  • videomix paper: https://arxiv.org/abs/2012.03457
  • pytorch-i3d(basic video transformation code): https://github.com/piergiaj/pytorch-i3d

Tubemix is like a 'video-in-video' augmentation, and Stackmix is 'video-to-video'. Use videomix.py for implementing Tubemix and Stackmix.

rgb "TableTennis"	rgb "Archery"

rgb stackmix	rgb tubemix

opt flow-u "TableTennis"	opt flow-u "Archery"

opt flow-u stackmix	opt flow-u tubemix

Training on I3D with Stackmix and Tubemix augmentation.

* RCRF represents applying random crop and random flipping Probability of implementing Stackmix or Tubemix is fixed to p=0.5.

I've also explored how beta distribution effect on training accuracy. Figure below shows the PDF of &\lambda$~beta(α,α).

• UCF-101

• HMDB-51

Apply on UCF-101, Stackmix and Tubemix could derive performance improvement(+1~2%) with UCF-101 datasets in both streams. However, when applied to HMDB-51, the performance of temporal stream was improved, but the performance of spatial stream was rather reduced.

• Speed jittering augmentation is work in progress.