Facial Expression Recognition by De-expression Residue Learning

Yang, Huamin; Ciftci, Umur Aybars; Liu, Yin

doi:10.1109/cvpr.2018.00231

Cited by 359 publications

(216 citation statements)

References 24 publications

Supporting

Mentioning

197

Contrasting

Unclassified

Order By: Relevance

“…Generative adversarial networks (GANs) can be exploited to solve this issue by frontalizing face images while preserving expression characteristics [180] or synthesizing arbitrary poses to help train the pose-invariant network [181]. Another advantage of GANs is that the identity variations can be explicitly disentangled through generating the corresponding neutral face image [141] or synthesizing different expressions while preserving the identity information for identityinvariant FER [182]. Moreover, GANs can help augment the training data on both size and diversity.…”

Section: Discussionmentioning

confidence: 99%

“…Chen et al [183] proposed a Privacy-Preserving Representation-Learning Variational GAN (PPRL-VGAN) that combines VAE and GAN to learn an identity-invariant representation that is explicitly disentangled from the identity information and generative for expression-preserving face image synthesis. Yang et al [141] proposed a De-expression Residue Learning (DeRL) procedure to explore the expressive information, which is filtered out during the de-expression process but still embedded in the generator. Then the model extracted this information from the generator directly to mitigate the influence of subject variations and improve the FER performance.…”

Section: Generative Adversarial Network (Gans)mentioning

confidence: 99%

See 1 more Smart Citation

Deep Facial Expression Recognition: A Survey

Deng

2022

IEEE Trans. Affective Comput.

1,015

593

View full text Add to dashboard Cite

With the transition of facial expression recognition (FER) from laboratory-controlled to challenging in-the-wild conditions and the recent success of deep learning techniques in various fields, deep neural networks have increasingly been leveraged to learn discriminative representations for automatic FER. Recent deep FER systems generally focus on two important issues: overfitting caused by a lack of sufficient training data and expression-unrelated variations, such as illumination, head pose and identity bias. In this paper, we provide a comprehensive survey on deep FER, including datasets and algorithms that provide insights into these intrinsic problems. First, we introduce the available datasets that are widely used in the literature and provide accepted data selection and evaluation principles for these datasets. We then describe the standard pipeline of a deep FER system with the related background knowledge and suggestions of applicable implementations for each stage. For the state of the art in deep FER, we review existing novel deep neural networks and related training strategies that are designed for FER based on both static images and dynamic image sequences, and discuss their advantages and limitations. Competitive performances on widely used benchmarks are also summarized in this section. We then extend our survey to additional related issues and application scenarios. Finally, we review the remaining challenges and corresponding opportunities in this field as well as future directions for the design of robust deep FER systems.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Generative Adversarial Network (Gans)mentioning

confidence: 99%

Deep Facial Expression Recognition: A Survey

Deng

2022

IEEE Trans. Affective Comput.

1,015

593

View full text Add to dashboard Cite

show abstract

“…Jeon constructed a real-time facial expression recognizer using a deep neural network which is invariant to the subject. Soon after, many deep learning methods are used for facial expression recognition and have achieved good performance [11][12][13][14]. In summary, there are a variety of facial expression recognition methods, but the method of expression intensity based on deep learning is less [15].…”

Section: Related Workmentioning

confidence: 99%

Personalized Clothing Recommendation Based on User Emotional Analysis

Gao

Ren

et al. 2020

Discrete Dynamics in Nature and Society

View full text Add to dashboard Cite

With the continuous development of economy, consumers pay more attention to the demand for personalization clothing. However, the recommendation quality of the existing clothing recommendation system is not enough to meet the user’s needs. When browsing online clothing, facial expression is the salient information to understand the user’s preference. In this paper, we propose a novel method to automatically personalize clothing recommendation based on user emotional analysis. Firstly, the facial expression is classified by multiclass SVM. Next, the user’s multi-interest value is calculated using expression intensity that is obtained by hybrid RCNN. Finally, the multi-interest value is fused to carry out personalized recommendation. The experimental results show that the proposed method achieves a significant improvement over other algorithms.

show abstract

“…The performance of CNN models trained on this dataset are greater than 95% due to the fact that the images are captured in a controlled environment (lab) and the emotions are overacted. The best performance was reached by Yang et al with a performance of 97.3% [20]. The dataset contains 593 video frames from 123 subjects.…”

Section: Introductionmentioning

confidence: 99%

Facial Expressions Recognition for Human–Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer

Melinte

Vlădăreanu

2020

Sensors

View full text Add to dashboard Cite

The interaction between humans and an NAO robot using deep convolutional neural networks (CNN) is presented in this paper based on an innovative end-to-end pipeline method that applies two optimized CNNs, one for face recognition (FR) and another one for the facial expression recognition (FER) in order to obtain real-time inference speed for the entire process. Two different models for FR are considered, one known to be very accurate, but has low inference speed (faster region-based convolutional neural network), and one that is not as accurate but has high inference speed (single shot detector convolutional neural network). For emotion recognition transfer learning and fine-tuning of three CNN models (VGG, Inception V3 and ResNet) has been used. The overall results show that single shot detector convolutional neural network (SSD CNN) and faster region-based convolutional neural network (Faster R-CNN) models for face detection share almost the same accuracy: 97.8% for Faster R-CNN on PASCAL visual object classes (PASCAL VOCs) evaluation metrics and 97.42% for SSD Inception. In terms of FER, ResNet obtained the highest training accuracy (90.14%), while the visual geometry group (VGG) network had 87% accuracy and Inception V3 reached 81%. The results show improvements over 10% when using two serialized CNN, instead of using only the FER CNN, while the recent optimization model, called rectified adaptive moment optimization (RAdam), lead to a better generalization and accuracy improvement of 3%-4% on each emotion recognition CNN.

show abstract

Facial Expression Recognition by De-expression Residue Learning

Cited by 359 publications

References 24 publications

Deep Facial Expression Recognition: A Survey

Deep Facial Expression Recognition: A Survey

Personalized Clothing Recommendation Based on User Emotional Analysis

Facial Expressions Recognition for Human–Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer

Contact Info

Product

Resources

About