This project is a part of our work, "Dynamic Vision Sensors for Human Activity Recognition" - Stefanie Anna Baby, Bimal Vinod, Chaitanya Chinni, Kaushik Mitra, accepted at the 4th IAPR Asian Conference on Pattern Recognition (ACPR) 2017.

The links to our work can be found at IEEE, arXiv and webpage.

The IITM DVS128 Gesture Dataset contains 10 hand gestures by 12 subjects with 10 sets each totalling to a 1200 hand gestures. These gestures are captured using a DVS128 camera.

The aedat dataset and the corresponding converted avi dataset can be downloaded from IITM_DVS_10.

.
├── lib
│   ├── dense_trajectory_release_v1.2
│   │   ├── ...
│   │   └── ...
│   ├── bov_encode.m
│   ├── generate_codebook.m
│   ├── generate_motion_maps.m
│   ├── groupfile_indices.m
│   ├── groupfile_indices_mm.m
│   ├── hidden_indices.m
│   ├── normalize_image.m
│   ├── parsave.m
│   ├── svm_loo.m
│   └── run_dense.sh
├── README.md
├── extract_features.m
├── run_dvs.m
└── startup.m

The lib folder contains a bunch of utility snippets used by the main code files.

The lib/dense_trajectory_release_v1.2 code is taken from http://lear.inrialpes.fr/people/wang/dense_trajectories which was from the work by "Action Recognition by Dense Trajectories" by Wang et al.. More details can be found in the README file associated with it.

The data folder should contain all the data in the root directory with the following structure. The extracted DVS data should be put in data/<dataset_name>/original_data with individual folders for each class.

The code expects .avi files and not aedat. All the features (motion-maps and dense-trajectories) are extracted by extract_features.m into the data/<dataset_name>/features_extracted folder.

The run_dvs.m program automatically segregates the test and train data into data/<dataset_name>/encoded_data for K-fold cross-validation.

The same structure is followed for any other dataset. The <dataset_name> can be IITM_DVS_10 or UCF11_DVS etc.

.
├── data
│   ├── IITM_DVS_10
│   │   ├── original_data
│   │   │   ├── comeHere
│   │   │   │   └── *.avi
│   │   │   ├── left_swipe
│   │   │   │   └── *.avi
│   │   │   └── ...
│   │   ├── features_extracted
│   │   │   └── ...
│   │   └── encoded_data
│   │   │   ├── test
│   │   │   └── train
│   ├── UCF11_DVS
│   │   ├── ...
│   │   └── ...
│   └── ...
└── ...

If you find our work helpful in your publications or projects, please consider citing our paper.

S. A. Baby and B. Vinod and C. Chinni and K. Mitra, "Dynamic Vision Sensors for Human Activity Recognition," 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), Nanjing, China, 2017.

@inproceedings{sababy:hardvs:2017,
    author={S. A. {Baby} and B. {Vinod} and C. {Chinni} and K. {Mitra}},
    booktitle={2017 4th IAPR Asian Conference on Pattern Recognition (ACPR)},
    title={Dynamic Vision Sensors for Human Activity Recognition},
    year={2017},
    pages={316-321},
    doi={10.1109/ACPR.2017.136},
    ISSN={2327-0985},
    month={Nov}
}

Thanks to Heng Wang for the dense-trajectories code - http://lear.inrialpes.fr/people/wang/dense_trajectories