Facial expression recognition techniques: a comprehensive survey

Abstract

Over the past decades, facial expression recognition (FER) has become an interesting research area and achieved substantial progress in computer vision. FER is to detect human emotional state related to biometric traits. Developing a machine based human FER system is a quite challenging task. Various FER systems are developed by analysing facial muscle motion and skin deformation based algorithms. In conventional FER system, the developed algorithms work on the constrained database. In the unconstrained environment, the efficacy of existing algorithms is limited due to certain issues during image acquisition. This study presents a detailed study on FER techniques, classifiers and datasets used for analysing the efficacy of the recognition techniques. Moreover, this survey will assist researchers in understanding the strategies and innovative methods that address the issues in a real-time application. Finally, the review presents the challenges encountered by FER system along with the future direction.

Fusing Visual Attention CNN and Bag of Visual Words for Cross-Corpus Speech Emotion Recognition

“…Emotion recognition (ER) plays an important role in human-computer interaction (HCI) [ 1 ]. During the last few years, numerous approaches have been proposed using different modalities (e.g., speech, facial expressions, and gestures) [ 2 , 3 , 4 , 5 ]. Speech is a useful modality in HCI research because of its different strengths, tremors, and speech rates depending on the emotional state.…”

Feature Selection on 2D and 3D Geometric Features to Improve Facial Expression Recognition

“…Most facial emotional recognition (FER) approaches follow three main steps: (a) face and landmark detection, (b) feature extraction, and (c) classification [ 4 , 5 , 6 ].…”

“…Sometimes it can include preprocessing to reduce the noise, image enhancement, and normalization. However, the most important task is face detection [ 6 ]. Once this is done, features are located on the face.…”

“…The second step is feature extraction. The objective is to extract an effective and efficient representation [ 6 ], methods such as feature point, interpolation polynomials, local binary patterns, Gabor, Histogram of Oriented Gradients, and optical flow are used [ 4 , 7 ]. It is important to remark that this step can include feature selection or feature construction.…”

“…FER systems still face many challenges like recognizing spontaneous expressions in real-time and in unconstrained environments. This includes variations in skin color; changes in illumination; aging; head pose variation; occlusion of objects like scarf, hair, or glasses; and complex background [ 6 ].…”