OAENet: Oriented attention ensemble for accurate facial expression recognition

Wang, Zhengning; Zeng, Fanwei; Liu, Shuaicheng; Zeng, Bing

doi:10.1016/j.patcog.2020.107694

Cited by 64 publications

(33 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the many previous works, some works such as STRNN [ 42 ], LBVCNN [ 41 ], TPOEM [ 38 ], PHRNN-MSCNN [ 39 ], and SAANet [ 43 ] used image sequence as their experimental data, while others used a static image. Although Specific preprocessing [ 16 ], ALAW [ 22 ], Feature loss [ 28 ], OAENet [ 35 ], and S-DSRN [ 23 ] used seven expressions, contempt expression is replaced with neural. Note that, in order to make a fair comparison, seven basic expressions with and without neutral are used for our experiment.…”

Section: Methodsmentioning

confidence: 99%

“…Mollahosseini et al [ 44 ] presented a new deep neural network (DNN) architecture to deal with the FER problem across seven well-known facial expression databases; the DNN has a good generalizability and accuracy. Wang et al [ 35 ] proposed an oriented attention pseudo-siamese network that consists of a maintenance branch and an attention branch, this network not only grabs a global picture but can also concentrate on important local areas. Generally speaking, deep learning methods perform better than traditional ones, so deep learning methods have gradually become mainstream.…”

Section: Related Workmentioning

confidence: 99%

“…Humans have the ability to quickly filter out irrelevant information and lock in parts of interest when recognizing objects. Recently, this kind of attention mechanism has been successfully applied in FER [ 17 , 18 , 20 , 21 , 24 , 25 , 26 , 29 , 35 , 43 , 57 ]. Zhong et al [ 21 ] divided a facial image into nonoverlapped patches to discover the common and specific patches that are important to discriminate all the expressions and only a particular expression, respectively; then, they discussed how different numbers of patches affect recognition performance.…”

Section: Related Workmentioning

confidence: 99%

“…Facial expressions can be divided into six basic emotions, namely, anger (An); disgust (Di); fear (Fe); happiness (Ha); sadness (Sa); surprise (Su); and one neutral (Ne) emotion [ 9 ], contempt (Co), was subsequently added as one of the basic emotions [ 10 ]. Recognition of these emotions can be categorized into image-based [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 ] and video-based [ 38 , 39 , 40 , 41 , 42 , 43 ] approaches. Image-based approaches only use information about the static input image to determine the category of facial expression; on the other hand, except when the spatial features extracted from a static image are available, video-based approaches can also use temporal information of a dynamic image sequence to capture the temporal changes of facial appearance when some facial expression occurs.…”

Section: Introductionmentioning

confidence: 99%

“…FER can also be divided into the traditional method [ 15 , 27 , 30 , 31 , 32 , 38 , 40 ], deep learning method [ 16 , 17 , 18 , 20 , 21 , 23 , 24 , 25 , 26 , 35 , 36 , 39 , 41 , 42 , 43 ], or a combination of the two [ 11 , 12 , 22 , 28 , 29 , 33 , 37 ]. Traditional FER systems usually involve facial representation and expression classification.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Patch Attention Layer of Embedding Handcrafted Features in CNN for Facial Expression Recognition

Liang

Liu

et al. 2021

Sensors

View full text Add to dashboard Cite

Recognizing facial expression has attracted much more attention due to its broad range of applications in human–computer interaction systems. Although facial representation is crucial to final recognition accuracy, traditional handcrafted representations only reflect shallow characteristics and it is uncertain whether the convolutional layer can extract better ones. In addition, the policy that weights are shared across a whole image is improper for structured face images. To overcome such limitations, a novel method based on patches of interest, the Patch Attention Layer (PAL) of embedding handcrafted features, is proposed to learn the local shallow facial features of each patch on face images. Firstly, a handcrafted feature, Gabor surface feature (GSF), is extracted by convolving the input face image with a set of predefined Gabor filters. Secondly, the generated feature is segmented as nonoverlapped patches that can capture local shallow features by the strategy of using different local patches with different filters. Then, the weighted shallow features are fed into the remaining convolutional layers to capture high-level features. Our method can be carried out directly on a static image without facial landmark information, and the preprocessing step is very simple. Experiments on four databases show that our method achieved very competitive performance (Extended Cohn–Kanade database (CK+): 98.93%; Oulu-CASIA: 97.57%; Japanese Female Facial Expressions database (JAFFE): 93.38%; and RAF-DB: 86.8%) compared to other state-of-the-art methods.

show abstract