Sample awareness-based personalized facial expression recognition

Li, Huihui; Wen, Gongjian

doi:10.1007/s10489-019-01427-2

Cited by 21 publications

(3 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It turns out Methods Years Accuracy(%) Conv+Inception [30] 2016 66.40 SMFER [31] 2018 65.12 GoogleNet [32] 2018 65.20 VGG+SVM [33] 2019 66.31 SAP [34] 2019 71.08 E-FCNN [28] 2021 66.17 DeepEmotion [35] 2021 uses transfer learning technique to overcome the shortage of training samples. For FER2013, the second-highest recognition accuracy of the test set is SAP [34], which is a sample awareness-based expression recognition method, in which a Bayesian classifier is used to select the most appropriate classifier from a set of candidate classifiers for the current test sample, and then the classifier is used to perform expression recognition on the current sample. For RAF-DB, the test set with the second-highest recognition accuracy is DACL [19], which combines center loss and an attention mechanism to selectively penalize features for enhanced discrimination.…”

Section: E Visualization Analysismentioning

confidence: 99%

RASN: Using Attention and Sharing Affinity Features to Address Sample Imbalance in Facial Expression Recognition

Yang

Kuang

et al. 2022

IEEE Access

View full text Add to dashboard Cite

The sample imbalance of expression datasets always leads to poor recognition results for minority classes. To solve this problem, we propose a facial expression recognition network, called Residual Attentive Sharing Network (RASN). There is a fact that different expressions have affinity features, which makes it possible for the minority classes to benefit from the majority classes in the expression feature extraction process, from which we propose a sharing affinity features module to compensate for the inadequate feature learning of minority classes by sharing affinity features. In addition, an affinity features attention module is added to highlight expression-related affinity features and suppress expression-unrelated ones for enhancing the role of sharing affinity features. Experiments on the CK+, RAF-DB, and FER2013 datasets validate the robustness of our method to sample imbalance. The validation accuracies of our method are 96.97% on CK+, 71.44% on FER2013, and 90.91% on RAF-DB, respectively, which exceed current state-of-the-art methods.

show abstract

Section: E Visualization Analysismentioning

confidence: 99%

RASN: Using Attention and Sharing Affinity Features to Address Sample Imbalance in Facial Expression Recognition

Yang

Kuang

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The FER pipeline proposed by Kumar and Rajagopal [43] has used normalized minimal feature vectors and semi-supervised Twin Support Vector Machine (TWSVM) learning. Li and Wen [44], proposed a sample awareness-based personalized (SAP) FER method that uses the Bayesian learning method to select the optimal classifier from the global perspective and then used the selected classifier to identify the emotional class of each test sample. The authors in [45] proposed a novel sparse modified Marginal Fisher analysis (SMMFA) for the FER task.…”

Section: Related Workmentioning

confidence: 99%

Facial Expression Recognition Using Dynamic Local Ternary Patterns With Kernel Extreme Learning Machine Classifier

2021

View full text Add to dashboard Cite

Rapid growth in advanced human-computer interaction (HCI) based applications has led to the immense popularity of facial expression recognition (FER) research among computer vision and pattern recognition researchers. Lately, a robust texture descriptor named Dynamic Local Ternary Pattern (DLTP) developed for face liveness detection has proved to be very useful in preserving facial texture information. The findings motivated us to investigate DLTP in more detail and examine its usefulness in the FER task. To this end, a FER pipeline is developed, which uses a sequence of steps to detect possible facial expressions in a given input image. Given an input image, the pipeline first locates and registers faces in it. In the next step, using an image enhancement operator, the FER pipeline enhances the facial images. Afterward, from the enhanced images, facial features are extracted using the DLTP descriptor. Subsequently, the pipeline reduces dimensions of the high-dimensional DLTP features via Principal Component Analysis (PCA). Finally, using the multi-class Kernel Extreme Learning Machine (K-ELM) classifier, the proposed FER scheme classifies the features into facial expressions. Extensive experiments performed on four in-the-lab and one in-the-wild FER datasets confirmed the superiority of the method. Besides, the cross-dataset experiments performed on different combinations of the FER datasets revealed its robustness. Comparison results with several state-of-the-art FER methods demonstrate the usefulness of the proposed FER scheme. The pipeline with a recognition accuracy of 99.76%, 99.72%, 93.98%, 96.71%, and 78.75%, respectively, on the CK+, RaF, KDEF, JAFFE, and RAF-DB datasets, outperformed the previous state-of-the-art.INDEX TERMS Facial expression recognition, dynamic local ternary pattern, principal component analysis, kernel extreme learning machine, cross-dataset, cross-validation.

show abstract

“…It is a spontaneous expression. e above properties of microexpressions make it a window to understand human real feelings [4,5]. erefore, microexpressions have many potential applications, such as criminal investigation, national defense security, clinical diagnosis, and humancomputer interaction.…”

Section: Introductionmentioning

confidence: 99%

Enhanced Forest Microexpression Recognition Based on Optical Flow Direction Histogram and Deep Multiview Network

Wang

2020

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In order to recognize the instantaneous changes of facial microexpressions in natural environment, a method based on optical flow direction histogram and depth multiview network to enhance forest microexpression recognition was proposed. In the preprocessing stage, the histogram equalization of the acquired face image is performed, and then the dense key points of the face are detected. According to the coordinates of the key points and the face action coding system (FACS), the face region is divided into 15 regions of interest (ROI). In the feature extraction stage, the optical flow direction histogram feature between adjacent frames in ROI is extracted to detect the peak frame of microexpression sequence. Finally, the average optical flow direction histogram feature of the image sequence from the initial frame to the peak frame is extracted. In the classification stage, firstly, the head pose parameters under horizontal degrees of freedom are estimated to eliminate the influence of head pose motion, and a forest multiview conditional probability model based on deep multiview network is established. Conditional probability and neural connection function are introduced into the node splitting learning of random tree to improve the learning ability and distinguishing ability of the model on the limited training set. Finally, multiview-weighted voting is used to determine the categories of facial microexpressions. Experiments on CASME II microexpression dataset show that the proposed method can effectively describe the changes of microexpressions and improve the recognition accuracy compared with other new methods.

show abstract

Sample awareness-based personalized facial expression recognition

Cited by 21 publications

References 62 publications

RASN: Using Attention and Sharing Affinity Features to Address Sample Imbalance in Facial Expression Recognition

RASN: Using Attention and Sharing Affinity Features to Address Sample Imbalance in Facial Expression Recognition

Facial Expression Recognition Using Dynamic Local Ternary Patterns With Kernel Extreme Learning Machine Classifier

Enhanced Forest Microexpression Recognition Based on Optical Flow Direction Histogram and Deep Multiview Network

Contact Info

Product

Resources

About