Robust Facial Landmark Detection via a Fully-Convolutional Local-Global Context Network
Facial landmarks are used to localize and represent salient regions of the face, such as: Eyes, Eyebrows, Nose, Mouth, Jawline
Challenges: different shapes, poses, lighting conditions, occlusions, etc.
Uses of landmark detection: face alignment, head pose estimation, face swapping, blink detection and much more.
Following is an iPython Notebook implementation of paper:
Robust Facial Landmark Detection via a Fully-Convolutional Local-Global Context Network, Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2018 - Daniel Merget, Matthias Rock, Gerhard Rigoll
Fully convolutional NN are good at modeling local features, but it results to constrained receptive field (local context).
To overcome this there are many ways: cascades/pooling etc. This paper proposes a new approach to use channel-wise/kernel convolution and dilated convolution (global context) to achieve the same with better accuracy than several SOTA methods. It introduces global context into a fully-convolutional neural network directly.
Major Contributions:
Uses Kernel Convolution directly within the network
Uses Dilated Convolutions to increase receptive field
Doesn’t depend on prior face detections
Input image is directly mapped to heatmap based tensor allowing the network to be accurate and robust
To run the code
main.py is meant to be run on Google Colab with Python3 & GPU runtime
main.ipynb is to see working output of main.py
Our Work-Plan
Our plan is to port their code to python so that it can be run on Google Colab for our experimentation and then later make it production ready
Steps to be followed:
Port CNTK neural network code to Keras
Port Matlab post-processing code to Python
Train the network using 300-W dataset
Make it work for multiple-faces (images from wild)
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent