Facial Expression Recognition Using Pose-Guided Face Alignment and Discriminative Features Based on Deep Learning

Li, Jun; Feng, Yanjun; Wang, Hongxia

doi:10.1109/access.2021.3078258

Cited by 25 publications

(13 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FER in General: FER has been studied widely and a broad variety of approaches are proposed. In the following, we review some of the important work such as [8]- [19] in this context.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Fard

Mahoor

2022

IEEE Access

View full text Add to dashboard Cite

Automated Facial Expression Recognition (FER) in the wild using deep neural networks is still challenging due to intra-class variations and inter-class similarities in facial images. Deep Metric Learning (DML) is among the widely used methods to deal with these issues by improving the discriminative power of the learned embedded features. This paper proposes an Adaptive Correlation (Ad-Corre) Loss to guide the network towards generating embedded feature vectors with high correlation for within-class samples and less correlation for between-class samples. Ad-Corre consists of 3 components called Feature Discriminator, Mean Discriminator, and Embedding Discriminator. We design the Feature Discriminator component to guide the network to create the embedded feature vectors to be highly correlated if they belong to a similar class, and less correlated if they belong to different classes. In addition, the Mean Discriminator component leads the network to make the mean embedded feature vectors of different classes to be less similar to each other. We use Xception network as the backbone of our model, and contrary to previous work, we propose an embedding feature space that contains k feature vectors. Then, the Embedding Discriminator component penalizes the network to generate the embedded feature vectors, which are dissimilar. We trained our model using the combination of our proposed loss functions called Ad-Corre Loss jointly with the cross-entropy loss. We achieved a very promising recognition accuracy on AffectNet, RAF-DB, and FER-2013. Our extensive experiments and ablation study indicate the power of our method to cope well with challenging FER tasks in the wild.

show abstract

“…FER in General: FER has been studied widely and a broad variety of approaches are proposed. In the following, we review some of the important work such as [8]- [19] in this context.…”

Section: Related Workmentioning

confidence: 99%

“…Also, their proposed architecture is designed to use the attention mechanism to deal with part location and feature fusion problems. Liu et al [19] proposed a framework including a face alignment method to reduce the intra-class difference, a feature extraction module to obtain the semantic information, and a backbone model for FER.…”

Section: Related Workmentioning

confidence: 99%

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Fard

Mahoor

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Banerjee et al [22] proposed an ensemble of selected features of several CNNs based on a twostage feature selection algorithm, namely fuzzy entropy (FE) and total contribution score (TCS) for erythrocytes detection. Liu et al [23] proposed a deep ensemble model for facial expression recognition. A hybrid feature representation method was used to acquire high-level discriminative features and a lightweight backbone fusion based on VGG16 and ResNet was constructed to achieve low-calculation training.…”

Section: Related Workmentioning

confidence: 99%

GFF-CARVING: Graph Feature Fusion for the Recognition of Highly Varying and Complex Balinese Carving Motifs

2022

View full text Add to dashboard Cite

The recognition of Balinese carving motifs is challenging due to the highly varying and interrelated motifs of Balinese carvings and in addition to the scantiness of Balinese carving data. This study proposed a method named GFF-CARVING for the recognition of Balinese carving motifs. GFF-CARVING is a deep learning architecture based on the Graph Convolutional Network (GCN) and Convolutional Neural Network (CNN) to extract image and graph features. GFF-CARVING applies feature fusion to improve the discriminative ability of the model to overcome these challenges and therefore improve its recognition performance. The proposed method consists of three main modules, the image representation learning module, the graph representation learning module, and the prediction module. The image representation learning module is based on ResNet and extracts the image features using global max pooling. The graph representation learning module is based on GCN and extracts the graph features. The graph features are handcrafted features that are built based on the occurrence relationship between the constituent sub-motifs of Balinese carvings. The feature fusion generates new features that take into account the occurrence relationship between the sub-motifs. These new features are used in the prediction module to accurately recognize the Balinese carving motifs. Based on the experimental results, GFF-CARVING achieved the highest recognition accuracy of 98.93% compared to other state-of-the-art models. These results indicated that feature fusion based on the handcrafted graph features and image features improved the discriminative ability of GFF-CARVING in recognizing Balinese carving motifs.

show abstract

“…Dynamic-based input use time and motion information from sequences of images having facial expression. To escape from the traditional feature extraction process (Geometry based method, Template based method, and Appearance based method), the CNN model is used as a feature extractor for Emotion Detection using facial expression [13,14]. But Convolutional neural network is not enough for emotion detection using facial expressions because these models take a long training time.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning based Framework for Emotion Recognition using Facial Expression

Bukhari

Hussain

Ayoub

et al. 2022

PakJET

View full text Add to dashboard Cite

Human convey their message in different forms. Expressing their emotions and moods through their facial expression is one of them. In this work, to avoid the traditional process of feature extraction (Geometry based method, Template based method, and Appearance based method), CNN model is used as a feature extractor for emotion detection using facial expression. In this study we also used three pre-trained models VGG-16, ResNet-50, Inception-V3. This Experiment is done on Fer-2013 facial expression dataset and Cohn Extended (CK+) dataset. By using FER-2013 dataset the accuracy rates for CNN, ResNet-50, VGG-16 and Inception-V3 are 76.74%, 85.71%, 85.78%s, 97.93% respectively. Similarly, the experimental results using CK+ dataset showed the accuracy rates for CNN, ResNet- 50, VGG-16 and Inception-V3 are 84.18%, 92.91%, 91.07%, and 73.16% respectively. The experimental results showed exceptional results for Inception-V3 with 97.93% using FER-2013 dataset and ResNet-50 with 91.92% using CK+ dataset.

show abstract

Facial Expression Recognition Using Pose-Guided Face Alignment and Discriminative Features Based on Deep Learning

Cited by 25 publications

References 59 publications

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

GFF-CARVING: Graph Feature Fusion for the Recognition of Highly Varying and Complex Balinese Carving Motifs

Deep Learning based Framework for Emotion Recognition using Facial Expression

Contact Info

Product

Resources

About