Two-level attention with two-stage multi-task learning for facial emotion recognition

Wang, Xiaohua; Peng, Muzi; Pan, Lijuan; Hu, Min; Jin, Chunhua; Ren, Fuji

doi:10.1016/j.jvcir.2019.05.009

Cited by 49 publications

(22 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CCC is used to compare the fitting degree of curve, and RMSE is sensitive to outliers. Our CCC performance is ordinary while RMSE performance is better, indicating that the generalization ability of the model is better than the Reference [17]. Barros et al[31] uses a neural model based on conditional antagonistic auto-encoder to perform the continuous emotional estimation.…”

Section: Framework Performance Experimental Results Arementioning

confidence: 85%

“…According to Ref. [17], we use the entire neuronal layer of each model as the feature instead of two values and optimize their weights at the same time, thus, making better use of the representation relations of different models. Then predict valence and arousal two values.…”

Section: System Structurementioning

confidence: 99%

“…This practice effectively solves the problem of shifting from training to easily trained tasks. According to Ref [17],. we use the entire neuronal layer of each model as the feature instead of two values and optimize their weights at the same time, thus, making better use of the representation relations of different models.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Multi‐Task and Attention Collaborative Network for Facial Emotion Recognition

Wang

et al. 2021

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

Facial expression is one of the most direct and effective ways to recognize emotions, widely used in human‐computer interaction, affective computing, and other research fields. Expression recognition can be divided into discrete expression classification and continuous dimensional emotion recognition. Most of the existing multi‐dimensional emotional estimation only considers the data under laboratory conditions. In this paper, facial emotion estimation is performed based on real‐world images and combined with the advantages of multi‐task learning and attention mechanism. We improve the multi‐task attention network (MTAN) from two aspects: task and feature. At the aspect of the task, the multi‐task collaborative attention network (MTCAN), which is based on task correlation, is proposed to solve task deviation in multi‐task learning. At the aspect of the feature, based on MTCAN, we came up with MTACN, which used the self‐attention mechanism to measure the importance of each attention module for each specific task. Then, we can capture the local‐to‐global connection in one step and fully exploit the feature within different levels of each task. Experimental results on the AffectNet dataset show that the performance of the model is significantly better than the original network, and the Root‐mean‐square error and consistency correlation coefficient results are superior to other existing models. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

show abstract

Section: Framework Performance Experimental Results Arementioning

confidence: 85%

Section: System Structurementioning

confidence: 99%

See 1 more Smart Citation

Multi‐Task and Attention Collaborative Network for Facial Emotion Recognition

Wang

et al. 2021

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

show abstract

“…Also, Ngo, et al [20] use deep transfer learning techniques by using a squeeze-and-excitation network (SENet) model SE-ResNet-50 which pretrained for using the largest dataset for human face VGGFace2 and proposes a new loss function and named weighted-cluster loss. Also W. Xiaohua, et al [21] propose a two-level attention network for facial expression recognition in a static image, the first level used to extract the position of features while the second level is a Bidirectional Recurrent Neural Network for utilizing the relation between all features between all layers.…”

Section: Related Workmentioning

confidence: 99%

Facial Expressions Recognition Via CNNCraft-net for Static RGB Images

Mostafa¹,

El‐Sayed²,

Belal³

2021

IJIES

View full text Add to dashboard Cite

Facial Expression Recognition (FER) is one of the most important research problems in computer vision and Artificial Intelligence (AI) due to its potential applications, many studies were proposed for the FER, whether based on using handcrafted (Craft) features with traditional machine learning techniques or using end to end convolution neural network (CNN). In this paper, we proposed a new model called CNNCraft-net based on combining the advantages of CNN and traditional models by concatenating features outputs from CNN, autoencoder, and handcrafted features such as scale-invariant feature transform (SIFT), speed up robust feature (SURF) and Oriented Fast Rotated Brief (ORB), computed by the bag of visual words (BOVW) to recognize eight facial expressions for static RGB images. For the comparative analysis, multiple metrics were used such as Accuracy, Loss, F-measure, precision, and recall. The high imbalanced AffectNet and FER2013 datasets were used to evaluate the proposed model where the proposed model achieves accuracy 61.9% for eight expressions and 65% for seven expressions for AffectNet and 69% for FER2013.

show abstract

“…The conventional facial recognition system with PCA is really a simple face recognition approach and data compression method. Still, the lighting conditions are not sensitive [12]. The LDA has been one of the commonly utilized projection techniques which would be effective in mapping high-dimensional measurements into the low-dimensional space.…”

Section: Introductionmentioning

confidence: 99%

Optimization Assisted Convolutional Neural Network for Facial Emotion Recognition

Sarkar¹

2020

View full text Add to dashboard Cite

Facial Expression Recognition (FER) is an important type of visual information that can be used to understand a human"s emotional situation. FER has attained a significant interest in human-computer interaction, autopilot, medical healing as well as various face expression dependent areas, and it is enormously used in most research areas. Hence, this paper intends to develop an intelligent facial emotion recognition model by following two major processes namely (a) Feature extraction and (b) Classification. Initially, the input image is subjected to extract Local Binary Pattern (LBP) based features. Further, the extracted features are classified using a Convolutional neural network (CNN). Moreover, the weights of CNN are optimally tuned by the Improvised Steering angle and Gear-based ROA (ISG-ROA) algorithm. Finally, the superiority of the ISG-ROA method is compared over existing methods and its improvement is proved effective.

show abstract

Two-level attention with two-stage multi-task learning for facial emotion recognition

Cited by 49 publications

References 48 publications

Multi‐Task and Attention Collaborative Network for Facial Emotion Recognition

Multi‐Task and Attention Collaborative Network for Facial Emotion Recognition

Facial Expressions Recognition Via CNNCraft-net for Static RGB Images

Optimization Assisted Convolutional Neural Network for Facial Emotion Recognition

Contact Info

Product

Resources

About