By Yong Li, Jiabei Zeng, Shiguang Shan, Xilin Chen
News! 2019/02/25. We add the model config of paper: "Occlusion aware facial expression recognition using CNN with attention mechanism". All the details can be found in prototxt/gACNN_train.prototxt.
Code for our ICPR 2018 paper: "Patch-Gated CNN for Occlusion-aware Facial Expression Recognition". We designed a Patch-Gated CNN that can percept and ignore the occlusions for facial expression recognition. All the details can be found in prototxt/pACNN_train.prototxt.
Note that the code is based on caffe, a famous deep learning framework.
The order of the 68 facial landmarks can be found at Link.
The following figure shows how we select the ciritcal points to crop the patches from a facial image:
(a) denotes totally 30 landmarks within original 68 facial landmarks. These points are involved in point selection
(b) shows 16 points that we pick to cover the facial regions on or around eyes, eyebrows, nose, mouth
(c) illustrates four points that are recomputed to better cover eyes and eyebrows
(d) displays four points that are re-computed to cover facial cheeks
(e) shows the selected 24 facial landmarks, around which the patches in (f) are cropped
Training yourself:
We designed a caffe layer named as multi_roi_pooling_layer. Currently the layer is provided with a GPU version.
Building the multi_roi_pooling_layer with the related caffe.proto in proto folder, you can start training a model
immediately.
Precautions:
The training and testing image list should be arranged as:
image_path expression_label point1_h point1_w pint2_h point2_w ... point_24_h point24_w
An example of train.list has been provided in train_list_example folder.
We provided a python script to convert 68 facial landmarks to desired 24 points, the file locate in convert_point folder.
Dateset resource:
We collected and labelled a facial expression dataset (FED-RO) in the presence of real occlusions: . download link
Alternative download link based on Baidu Yun. Extraction code: o5di
@inproceedings{li2018patch,
title={Patch-Gated CNN for Occlusion-aware Facial Expression Recognition},
author={Li, Yong and Zeng, Jiabei and Shan, Shiguang and Chen, Xilin},
booktitle={ICPR},
year={2018}
}
@article{li2019occlusion,
title={Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism},
author={Li, Yong and Zeng, Jiabei and Shan, Shiguang and Chen, Xilin},
journal={IEEE Transactions on Image Processing},
volume={28},
number={5},
pages={2439--2450},
year={2019}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent