Neutral Face Classification Using Personalized Appearance Models for Fast and Robust Emotion Detection

Abstract

Facial expression recognition is one of the open problems in computer vision. Robust neutral face recognition in real time is a major challenge for various supervised learning-based facial expression recognition methods. This is due to the fact that supervised methods cannot accommodate all appearance variability across the faces with respect to race, pose, lighting, facial biases, and so on, in the limited amount of training data. Moreover, processing each and every frame to classify emotions is not required, as user stays neutral for majority of the time in usual applications like video chat or photo album/web browsing. Detecting neutral state at an early stage, thereby bypassing those frames from emotion classification would save the computational power. In this paper, we propose a light-weight neutral versus emotion classification engine, which acts as a pre-processer to the traditional supervised emotion classification approaches. It dynamically learns neutral appearance at key emotion (KE) points using a statistical texture model, constructed by a set of reference neutral frames for each user. The proposed method is made robust to various types of user head motions by accounting for affine distortions based on a statistical texture model. Robustness to dynamic shift of KE points is achieved by evaluating the similarities on a subset of neighborhood patches around each KE point using the prior information regarding the directionality of specific facial action units acting on the respective KE point. The proposed method, as a result, improves emotion recognition (ER) accuracy and simultaneously reduces computational complexity of the ER system, as validated on multiple databases.

A fast and robust emotion recognition system for real-world mobile phone data

“…For the more details on the method, refer [14]. This helps to skip analyzing neutral frames using intensive vision algorithms to detect emotions, and hence helps to achieve a real-time ER on a mobile platform.…”

Facial emotion recognition using min-max similarity classifier

“…The geometry-based methods extract main feature points and their shapes from the face image and calculate the distances between them. While, appearance-based methods focus on the face texture using different classification and template matching methods [14], [15]. In this paper, we focus on facial emotion recognition based on template matching techniques that remains a challenging task [16]- [18].…”

“…Facial emotion recognition accuracy depends on the robustness of a feature extraction method to intra-class variations and classifier performance under noisy conditions and with various types of occlusions [10]. Even thought a variety of approaches for the automated recognition of human expressions from the face images using template matching methods have been investigated and proposed over the last few years [14], the emotion recognition method with robust feature extraction and effective classification techniques accompanied by low computational complexity is still an open research problem [21]. Therefore, in this paper, we address the issues of matching templates through pixel normalization followed by the removal of inter-image feature outliers using a Min-Max similarity metric.…”

Facial emotion recognition using Min-Max similarity classifier

“…The geometry-based methods extract main feature points and their shapes from the face image and calculate the distances between them. While, appearance-based methods focus on the face texture using different classification and template matching methods Chiranjeevi et al (2015); Ghimire and Lee (2013). In this paper, we focus on facial emotion recognition based on template matching techniques that remains a challenging task Brunelli and Poggio (1993); Zhang et al (2011); Wang et al (2014).…”

“…Facial emotion recognition accuracy depends on the robustness of a feature extraction method to intraclass variations and classifier performance under noisy conditions and with various types of occlusions over the last few years Chiranjeevi et al (2015), the emotion recognition method with robust feature extraction and effective classification techniques accompanied by low computational complexity is still an open research problem Kamarol et al (2016). Therefore, in this paper, we address the issues of matching templates through pixel normalization followed by the removal of inter-image feature outliers using a Min-Max similarity metric.…”