Attention mechanism-based CNN for facial expression recognition

Li, Jing; Jin, Kuang; Zhou, Dalin; Kubota, Naoyuki; Ju, Zhaojie

doi:10.1016/j.neucom.2020.06.014

Cited by 193 publications

(41 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Table 5 , our method achieves better performance and shows high results on JAFFE for seven-class. The Attention-based CNN method [ 37 ], which features the highest accuracy of the methods shown in Table 5 , is not as good as ours for the CK+ and Oulu-CASIA. Note that the work [ 32 ] achieved an accuracy of 94.8% for six-class by a new face descriptor, namely, local directional ternary pattern; however, for seven-class, we achieved an 0.18% improvement compared to theirs.…”

Section: Methodsmentioning

confidence: 93%

“…Our PAL method achieves the best performance and outperforms the previous best video-based work SAANet [ 43 ] by 9.24%. For the image-based method, Attention-based CNN [ 37 ], our model outperforms it by 2.94%. The confusion matrix in Figure 4 b expresses that happiness expression is very easy to be recognized, while anger and sadness show relatively low performance.…”

Section: Methodsmentioning

confidence: 99%

“…Facial expressions can be divided into six basic emotions, namely, anger (An); disgust (Di); fear (Fe); happiness (Ha); sadness (Sa); surprise (Su); and one neutral (Ne) emotion [ 9 ], contempt (Co), was subsequently added as one of the basic emotions [ 10 ]. Recognition of these emotions can be categorized into image-based [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 ] and video-based [ 38 , 39 , 40 , 41 , 42 , 43 ] approaches. Image-based approaches only use information about the static input image to determine the category of facial expression; on the other hand, except when the spatial features extracted from a static image are available, video-based approaches can also use temporal information of a dynamic image sequence to capture the temporal changes of facial appearance when some facial expression occurs.…”

Section: Introductionmentioning

confidence: 99%

“…FER can also be divided into the traditional method [ 15 , 27 , 30 , 31 , 32 , 38 , 40 ], deep learning method [ 16 , 17 , 18 , 20 , 21 , 23 , 24 , 25 , 26 , 35 , 36 , 39 , 41 , 42 , 43 ], or a combination of the two [ 11 , 12 , 22 , 28 , 29 , 33 , 37 ]. Traditional FER systems usually involve facial representation and expression classification.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Patch Attention Layer of Embedding Handcrafted Features in CNN for Facial Expression Recognition

Liang

Liu

et al. 2021

Sensors

View full text Add to dashboard Cite

Recognizing facial expression has attracted much more attention due to its broad range of applications in human–computer interaction systems. Although facial representation is crucial to final recognition accuracy, traditional handcrafted representations only reflect shallow characteristics and it is uncertain whether the convolutional layer can extract better ones. In addition, the policy that weights are shared across a whole image is improper for structured face images. To overcome such limitations, a novel method based on patches of interest, the Patch Attention Layer (PAL) of embedding handcrafted features, is proposed to learn the local shallow facial features of each patch on face images. Firstly, a handcrafted feature, Gabor surface feature (GSF), is extracted by convolving the input face image with a set of predefined Gabor filters. Secondly, the generated feature is segmented as nonoverlapped patches that can capture local shallow features by the strategy of using different local patches with different filters. Then, the weighted shallow features are fed into the remaining convolutional layers to capture high-level features. Our method can be carried out directly on a static image without facial landmark information, and the preprocessing step is very simple. Experiments on four databases show that our method achieved very competitive performance (Extended Cohn–Kanade database (CK+): 98.93%; Oulu-CASIA: 97.57%; Japanese Female Facial Expressions database (JAFFE): 93.38%; and RAF-DB: 86.8%) compared to other state-of-the-art methods.

show abstract

Section: Methodsmentioning

confidence: 93%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Patch Attention Layer of Embedding Handcrafted Features in CNN for Facial Expression Recognition

Liang

Liu

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The product of the extraction process affected the system's output [21]. The traditional algorithms for facial extraction can be divided into two categories: 1) geometric approaches such as Active Appearance Models (AAM); and 2) appearance-based methods like Gabor wavelet representation and Local Binary Pattern (LBP) [34]; in a geometric approach, various geometrical parameters such as position, angle, points of reference, etc. are considered.…”

Section: Facial Expression Recognition (Fer)mentioning

confidence: 99%

Facial Expression Recognition Based on Deep Learning Convolution Neural Network: A Review

Abdullah

Abdulazeez

2021

JSCDM

View full text Add to dashboard Cite

Facial emotional processing is one of the most important activities in effective calculations, engagement with people and computers, machine vision, video game testing, and consumer research. Facial expressions are a form of nonverbal communication, as they reveal a person's inner feelings and emotions. Extensive attention to Facial Expression Recognition (FER) has recently been received as facial expressions are considered. As the fastest communication medium of any kind of information. Facial expression recognition gives a better understanding of a person's thoughts or views and analyzes them with the currently trending deep learning methods. Accuracy rate sharply compared to traditional state-of-the-art systems. This article provides a brief overview of the different FER fields of application and publicly accessible databases used in FER and studies the latest and current reviews in FER using Convolution Neural Network (CNN) algorithms. Finally, it is observed that everyone reached good results, especially in terms of accuracy, with different rates, and using different data sets, which impacts the results.

show abstract

Expression Recognition Algorithm Based on Infrared Image

Qiu

2021

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

View full text Add to dashboard Cite

Attention mechanism-based CNN for facial expression recognition

Cited by 193 publications

References 36 publications

Patch Attention Layer of Embedding Handcrafted Features in CNN for Facial Expression Recognition

Patch Attention Layer of Embedding Handcrafted Features in CNN for Facial Expression Recognition

Facial Expression Recognition Based on Deep Learning Convolution Neural Network: A Review

Expression Recognition Algorithm Based on Infrared Image

Contact Info

Product

Resources

About