Deep multi-path convolutional neural network joint with salient region attention for facial expression recognition

Xie, Siyue; Hu, Haifeng; Wu, Yilin

doi:10.1016/j.patcog.2019.03.019

Cited by 183 publications

(51 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Facial expression recognition (FER) has always been a challenging topic in computer vision. Researchers usually aim to build a system that can identify different expressions in the images automatically [33]. Research on facial expression recognition relies heavily on an adequate dataset of facial expressions.…”

Section: Discussionmentioning

confidence: 99%

Feature Guided CNN for Baby’s Facial Expression Recognition

Lin

Jiang

2020

Complexity

View full text Add to dashboard Cite

State-of-the-art facial expression methods outperform human beings, especially, thanks to the success of convolutional neural networks (CNNs). However, most of the existing works focus mainly on analyzing an adult’s face and ignore the important problems: how can we recognize facial expression from a baby’s face image and how difficult is it? In this paper, we first introduce a new face image database, named BabyExp, which contains 12,000 images from babies younger than two years old, and each image is with one of three facial expressions (i.e., happy, sad, and normal). To the best of our knowledge, the proposed dataset is the first baby face dataset for analyzing a baby’s face image, which is complementary to the existing adult face datasets and can shed some light on exploring baby face analysis. We also propose a feature guided CNN method with a new loss function, called distance loss, to optimize interclass distance. In order to facilitate further research, we provide the benchmark of expression recognition on the BabyExp dataset. Experimental results show that the proposed network achieves the recognition accuracy of 87.90% on BabyExp.

show abstract

Section: Discussionmentioning

confidence: 99%

Feature Guided CNN for Baby’s Facial Expression Recognition

Lin

Jiang

2020

Complexity

View full text Add to dashboard Cite

show abstract

“…Although these approaches are effective methods in extracting spatial information, they fail to capture morphological and contextual variations in the expression process. Recent methods aim to solve this problem by using massive datasets to obtain more efficient features of FER [9][10][11][12][13][14][15]. Some researchers use multimodal fusion to recognize emotions, such as voices, expressions, and actions [16].…”

Section: Related Workmentioning

confidence: 99%

Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

Zhang

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

This paper addresses the problem of Facial Expression Recognition (FER), focusing on unobvious facial movements. Traditional methods often cause overfitting problems or incomplete information due to insufficient data and manual selection of features. Instead, our proposed network, which is called the Multi-features Cooperative Deep Convolutional Network (MC-DCN), maintains focus on the overall feature of the face and the trend of key parts. The processing of video data is the first stage. The method of ensemble of regression trees (ERT) is used to obtain the overall contour of the face. Then, the attention model is used to pick up the parts of face that are more susceptible to expressions. Under the combined effect of these two methods, the image which can be called a local feature map is obtained. After that, the video data are sent to MC-DCN, containing parallel sub-networks. While the overall spatiotemporal characteristics of facial expressions are obtained through the sequence of images, the selection of keys parts can better learn the changes in facial expressions brought about by subtle facial movements. By combining local features and global features, the proposed method can acquire more information, leading to better performance. The experimental results show that MC-DCN can achieve recognition rates of 95%, 78.6% and 78.3% on the three datasets SAVEE, MMI, and edited GEMEP, respectively.

show abstract

“…With the rapid development of deep learning, great progress has been made in pattern recognition. Much work has been done on facial expression recognition based on deep neural networks, and good results have been obtained [14][15][16][17][18][19]. At present, developing effective expression recognition based on a convolutional neural network is still a problem worthy of study.…”

Section: Introductionmentioning

confidence: 99%

A Facial Expression Recognition Method Based on a Multibranch Cross-Connection Convolutional Neural Network

2021

View full text Add to dashboard Cite

This paper proposes a new method based on a multiple branch cross-connected convolutional neural network (MBCC-CNN) for facial expression recognition. Compared with traditional machine learning methods, the proposed method can extract image features more effectively. In addition, in contrast to single-structure convolutional neural networks, the MBCC-CNN model is constructed based on the residual connection, Network in Network, and tree structure approaches together. It also adds a shortcut cross connection for the summation of the convolution output layer, which makes the data flow between networks more smooth, improves the feature extraction ability of each receptive field. The proposed method can fuse the features of each branches more effectively, which solves the problem of insufficient feature extraction of each branches and increases the recognition performance. The experimental results based on the Fer2013, CK+, FER+ and RAF data sets show that the recognition rates of the proposed MBCC-CNN method are 71.52%, 98.48%, 88.10% and 87.34%, respectively. Compared with some most recently work, the proposed method can provide better facial expression recognition performance and has good robustness. INDEX TERMS Facial expression recognition; Convolutional neural network; Residual connection; Network in Network; Robustness.

show abstract

Deep multi-path convolutional neural network joint with salient region attention for facial expression recognition

Cited by 183 publications

References 57 publications

Feature Guided CNN for Baby’s Facial Expression Recognition

Feature Guided CNN for Baby’s Facial Expression Recognition

Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

A Facial Expression Recognition Method Based on a Multibranch Cross-Connection Convolutional Neural Network

Contact Info

Product

Resources

About