Facial Expression Recognition with Fusion Features Extracted from Salient Facial Areas

Abstract

In the pattern recognition domain, deep architectures are currently widely used and they have achieved fine results. However, these deep architectures make particular demands, especially in terms of their requirement for big datasets and GPU. Aiming to gain better results without deep networks, we propose a simplified algorithm framework using fusion features extracted from the salient areas of faces. Furthermore, the proposed algorithm has achieved a better result than some deep architectures. For extracting more effective features, this paper firstly defines the salient areas on the faces. This paper normalizes the salient areas of the same location in the faces to the same size; therefore, it can extracts more similar features from different subjects. LBP and HOG features are extracted from the salient areas, fusion features’ dimensions are reduced by Principal Component Analysis (PCA) and we apply several classifiers to classify the six basic expressions at once. This paper proposes a salient areas definitude method which uses peak expressions frames compared with neutral faces. This paper also proposes and applies the idea of normalizing the salient areas to align the specific areas which express the different expressions. As a result, the salient areas found from different subjects are the same size. In addition, the gamma correction method is firstly applied on LBP features in our algorithm framework which improves our recognition rates significantly. By applying this algorithm framework, our research has gained state-of-the-art performances on CK+ database and JAFFE database.

Using deep autoencoders for facial expression recognition

“…Only 327 out of 593 sequences of images are given the labels for 7 human facial expressions. Out of 7 expressions, we used six expressions (i.e., angry, happy, disgust, sadness, surprise, and fear) similar to the methods adopted in [8], [18], [36]. Contempt has only 18 labeled images so it was not included in our experiment.…”

“…Recently, a comprehensive study has been made by Liu et al [8], they also combined HOG with Local Binary Patterns (LBP) features. For dimension reduction of extracted features, PCA was used.…”

“…PCA is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. PCA is an effective method to reduce the feature dimension and has been extensively being applied in FER for dimension reduction of features [5], [8]. Therefore, we chose PCA to compare its performance with nonlinear dimension reduction by autoencoder.…”

“…Due to the complexity of multi-view features, dimension reduction and more meaningful representation of this high dimensional data is a challenging task. Therefore, techniques like Principal Component Analysis (PCA) and Local Binary Pattern (LBP), [5], [8], Non-Negative Matrix Factorization (NMF), etc., are being used to overcome high dimensionality problem by representing the most relevant features in lowerdimensions.…”

Facial expression recognition using three-stage support vector machines

“…Others decided to combine several extraction methods and then speed up the computation time. Liu et al [6] combined LBP and HOG methods for feature extraction; then, they performed principal component analysis (PCA) to minimize the time and speed needed for computation, and, finally, they used SVM classification. However, some decided to implement deep learning rather than continuing to use machine learning.…”