Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

Abstract

This paper addresses the problem of Facial Expression Recognition (FER), focusing on unobvious facial movements. Traditional methods often cause overfitting problems or incomplete information due to insufficient data and manual selection of features. Instead, our proposed network, which is called the Multi-features Cooperative Deep Convolutional Network (MC-DCN), maintains focus on the overall feature of the face and the trend of key parts. The processing of video data is the first stage. The method of ensemble of regression trees (ERT) is used to obtain the overall contour of the face. Then, the attention model is used to pick up the parts of face that are more susceptible to expressions. Under the combined effect of these two methods, the image which can be called a local feature map is obtained. After that, the video data are sent to MC-DCN, containing parallel sub-networks. While the overall spatiotemporal characteristics of facial expressions are obtained through the sequence of images, the selection of keys parts can better learn the changes in facial expressions brought about by subtle facial movements. By combining local features and global features, the proposed method can acquire more information, leading to better performance. The experimental results show that MC-DCN can achieve recognition rates of 95%, 78.6% and 78.3% on the three datasets SAVEE, MMI, and edited GEMEP, respectively.

A Robust Facial Expression Recognition Algorithm Based on Multi-Rate Feature Fusion Scheme

“…3DIR [65] 93.21 STCNN-CRF [66] 93.04 CNN-CTSLSTM [67] 93.90 DDL [68] 99.16 DJSTN [26] 99.21 FDRL [69] 99.54 MC-DCN [70] 95.50 Proposed Scheme 96. 23 The results from the 10-times trial on the MMI dataset are in Table 12.…”

“…3DIR [65] 77.50 STCNN-CRF [66] 68.51 CNN-CTSLSTM [67] 78.4 DDL [68] 83.67 DJSTN [26] 87.88 FDRL [69] 85.23 MC-DCN [70] 78.6 Proposed Scheme 96.69…”

“…The accuracy comparisons of each method using the CK+ database is shown in Table 11. For the CK+ database, the proposed scheme which was denoted as the bold face, did not get the best result compared with some existing methods [26,[68][69][70].…”

Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information

“…Many studies on facial-image-based emotion recognition are being conducted, and there are similarities between emotion recognition and stress recognition studies since emotion and stress are related. Among studies on emotion recognition methods, many studies using facial landmark information are underway [41][42][43]. As changes in facial expressions are highly correlated with changes in facial landmarks, these studies input the coordinates of facial landmarks directly into a network or images created from facial landmarks.…”

A Method of Ground-Based Cloud Motion Predict: CCLSTM + SR-Net

“…Video-related image sequence predictions often have shorter time intervals between frames [24,25], and the sampling frequency is tens of times per second. The whole sky imager usually would not set such a high sampling frequency, which would result in excessive storage requirements.…”