Facial Expression Recognition with Fusion Features Extracted from Salient Facial Areas

Abstract

Abstract:In pattern recognition domain, deep architectures are widely used nowadays and they have achieved fine grades. However, these deep architectures need special demands, especially big datasets and GPU. Aiming to gain better grades without deep networks, we propose a simplified algorithm framework using fusion features extracted from the salient areas of faces. Furthermore, the proposed algorithm has achieved a better result than some deep architectures. For extracting more effective features, this paper firstly defines the salient areas on the faces. This paper normalizes the salient areas of the same location in the faces to the same size, therefore it can gain more similar features from different subjects. LBP and HOG features are extracted from the salient areas, fusion features' dimensions are reduced by Principal Component Analysis( PCA) and we apply softmax to classify the six basic expressions at once. This paper proposes a salient areas definitude method which uses peak expressions frames to compare with their neutral faces. This paper also proposes and applies the idea of normalizing the salient areas to align the specific areas which express the different expressions. This makes the salient areas found from different subjects have the same size. Besides, gamma correction method is firstly applied on LBP features in our algorithm framework which improves our recognition rates significantly. By applying this algorithm framework, our research has gained state-of-the-art performances on CK+ database and JAFFE database.

Novel multi‐scale deep residual attention network for facial expression recognition

“…The accuracy of the basic CNN algorithm using only the static feature was 84.48%. Liu et al [62] proposed a method to extract features of salient areas by using LBP and HOG features with gamma correction, which resulted in 90% accuracy. Goyani and Patel [63] used feature vectors by constructing the Haar wavelet of multiple levels with the face, eyes, and mouth.…”

“…In the future, we will continue to study and analyse the problem of FER, and expect to find better rules and strategies for superior performance. Table 4 Comparisons between our method and the state-of-the-arts FER algorithms on the JAFFE dataset Method Accuracy CNN [62] 84.48 salient feature [63] 90.00 multi-level Haar wavelet [64] 90.56 hierarchical deep learning method [58] 91.27 MCCNN [65] 95.80 Ms-RAN (ours) 96.03…”

Facial Expression Recognition with LBP and ORB Features

“…Ryu et al used a coarse grid for stable codes (highly related to nonexpression), and a finer one for active codes (highly related to expression) as the features for facial expression recognition [ 20 ]. Liu et al extracted LBP and HOG features from the salient areas that were defined on the faces; then Principal Component Analysis (PCA) is used to reduce the dimensions of the features which fused with LBP and HOG [ 21 ]. Ghimire et al divided the whole face region into domain-specific local regions; then region-specific appearance features and geometric features are extracted from the domain-specific regions for facial expression recognition [ 22 ].…”

Facial expression recognition techniques: a comprehensive survey

“…Liu et al [52] suggested a simple framework algorithm to extract facial features from the salient face area. The proposed algorithm normalises the salient features to the same size and extracts similar face features of different subjects.…”

“…SVM [41, 47, 48, 52, 53, 56, 58] is a well‐known supervised classification algorithm and also called a maximum margin classifier [61]. The hyperplane is selected with higher margin to discriminate two classes in n ‐dimensional space.…”