A curated list of facial expression recognition in both 7-emotion classification and affect estimation. In addition, this repository includes basic studies on FER and recent datasets.
👀 Update frequently!
⭐️: Importance of FER method (from 1 to 3 points)
🏆: Mandatory (research) paper to understand FER technology.
- [Jul. 2024] Add Conference and Journal papers.
- [Jun. 2024] Add some ArXiv and conference accepted papers.
- [Apr. 2024] Make
Multi-modal, EEG-based Emotion Recognition
section. Update papers. - [Mar. 2024] Update FG papers.
- [Feb. 2024] Update AAAI and ArXiv papers.
- [Jan. 2024] Update few ArXiv papers.
- [Nov. 2023] Update NeurIPS 2023 / ArXiv papers.
- [Oct. 2023] Update ACII 2023 paper / Fix details.
- [Sep. 2023] Update BMVC 2023 paper / Add
Dataset
section. - [Aug. 2023] Update ICCV 2023 / ACM MM 2023 papers.
- [Jul. 2023] Update AAAI 2023 paper.
- [Jun. 2023] Update ICML 2023 / ICASSP 2023 papers.
- [Apr. 2023] Update CVPR 2023 papers.
- [Mar. 2023] Initial update for FER papers.
- [Mar. 2023] Create awesome-SOTA-FER repository.
Please feel free to refer this repository for your FER research/development and send me pull requests or email Daeha Kim to add links.
Valence-arousal Affect Estimation and Analysis back-to-top
Facial Action Unit (AU) Detection (or Recognition) back-to-top
- This section also contains
landmark
detection.
Paper | Venue | Year | Impact | Code |
---|---|---|---|---|
Trend-Aware Supervision: On Learning Invariance for Semi-supervised Facial Action Unit Intensity Estimation | AAAI | 2024 | ⭐️⭐️ | N/A |
FAN-Trans: Online Knowledge Distillation for Facial Action Unit Detection | WACV | 2023 | ⭐️ | N/A |
Knowledge-Driven Self-Supervised Representation Learning for Facial Action Unit Recognition | CVPR | 2022 | ⭐️ | N/A |
Towards Accurate Facial Landmark Detection via Cascaded Transformers | CVPR | 2022 | ⭐️ | N/A |
Causal intervention for subject-deconfounded facial action unit recognition | AAAI (Oral) |
2022 | ⭐️⭐️⭐️ | N/A |
PIAP-DF: Pixel-Interested and Anti Person-Specific Facial Action Unit Detection Net with Discrete Feedback Learning | ICCV | 2021 | ⭐️ | N/A |
About Facial Privacy back-to-top
- Note: Some face recognition studies are included.
- You can get additional information here
Multi-modal, EEG-based Emotion Recognition back-to-top
Paper | Venue | Year | Impact | Code |
---|---|---|---|---|
Beyond Mimicking Under-Represented Emotions: Deep Data Augmentation with Emotional Subspace Constraints for EEG-Based Emotion Recognition | AAAI | 2024 | ⭐️⭐️ | N/A |
A Brain-Inspired Way of Reducing the Network Complexity via Concept-Regularized Coding for Emotion Recognition | AAAI | 2024 | ⭐️⭐️ | PyTorch |
Facial Expression Manipulation and Synthesis back-to-top
- Uprising
Talking face generation task
is included here!
Emotion Recognition, Facial Representations, and Others back-to-top
Datasets back-to-top
Name | Venue | Year | Impact | Site |
---|---|---|---|---|
FindingEmo: An Image Dataset for Emotion Recognition in the Wild | ArXiv | 2024 | ⭐️⭐️ | N/A |
VEATIC: Video-based Emotion and Affect Tracking in Context Dataset | WACV | 2024 | ⭐️ | Project |
EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes | ICCV | 2023 | ⭐️⭐️ | Project |
MimicME: A Large Scale Diverse 4D Database for Facial Expression Analysis | ECCV | 2022 | ⭐️⭐️ | Github |
CelebV-HQ: A Large-Scale Video Facial Attributes Dataset | ECCV | 2022 | ⭐️⭐️⭐️ | Project GitHub Demo |
FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videos | CVPR | 2022 | ⭐️⭐️⭐️ | Data |
MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wild | ACM MM | 2022 | ⭐️ | Site |
Aff-wild2: Extending the aff-wild database for affect recognition | ArXiv | 2018 | ⭐️⭐️ | Project |
Aff-wild: valence and arousal 'In-the-Wild' challenge | CVPRW | 2017 | ⭐️⭐️ | Project |
Challenges back-to-top
Name | Venue | Year | Site |
---|---|---|---|
Facial Expression Recognition and Analysis Challenge | FG | 2015 | Site |
Emotion Recognition in the Wild Challenge (EmotiW) | ICMI | 2013-2018 | Site |
Emotion Recognition in the Wild Challenge (EmotiW) | ICMI | 2023 | Site |
Affect Recognition in-the-wild: Uni/Multi-Modal Analysis & VA-AU-Expression Challenges | FG | 2020 | Site |
Affective Behavior Analysis In-the-Wild (1st) | FG | 2020 | Site |
Deepfake Detection Challenge | - | 2020 | Site Paper GitHub |
Affective Behavior Analysis In-the-Wild (2nd) | ICCV | 2021 | Site Paper |
Affective Behavior Analysis In-the-Wild (3rd) | CVPR | 2022 | Site Paper |
Affective Behavior Analysis In-the-Wild (4th) | ECCV | 2022 | Site |
The Multimodal Sentiment Analysis Challenge (MuSe) | ACM MM | 2022 | Site Paper |
Affective Behavior Analysis In-the-Wild (5th) | CVPR | 2023 | Site |
Emotionally And Culturally Intelligent AI (1st) | ICCV | 2023 | Site |
Affective Behavior Analysis in-the-wild (6th) | CVPR | 2024 | Site |
Affective Behavior Analysis in-the-wild (7th) | ECCV | 2024 | Site |
Tools back-to-top
Name | Paper(/Feature) | Site |
---|---|---|
FLAME | Learning a model of facial shape and expression from 4D scans | Project TensorFlow PyTorch |
FaceNet | Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks | PyTorch TensorFlow |
Landmark detection | - | PyTorch |
Age estimation | - | |
DataGen | APIs all about human faces and bodies | Web |
Remarkable Papers (2019~) back-to-top
Paper | Venue | Year | Impact | Code |
---|---|---|---|---|
A Compact Embedding for Facial Expression Similarity | CVPR | 2019 | ⭐️⭐️⭐️ | PyTorch |
A Personalized Affective Memory Model for Improving Emotion Recognition | ICML | 2019 | ⭐️⭐️ | TensorFlow |
Facial Expression Recognition via Relation-based Conditional Generative Adversarial Network | ICMI | 2019 | ⭐️ | N/A |
Facial Expression Recognition by De-expression Residue Learning | CVPR | 2018 | ⭐️⭐️ | N/A |
Joint pose and expression modeling for facial expression recognition | CVPR | 2018 | ⭐️⭐️ | TensorFlow |
Identity-Aware Convolutional Neural Network for Facial Expression Recognition | FG | 2017 | ⭐️⭐️⭐️ | N/A |
Facenet2expnet: Regularizing a deep face recognition net for expression recognition | FG | 2017 | ⭐️⭐️ | N/A |
Facial expression recognition from near-infrared videos | Image and Vision Computing (ELSEVIER) |
2011 | ⭐️ | N/A |
Facial expression decomposition | ICCV | 2003 | 🏆 | N/A |