Advances, Challenges, and Opportunities in Automatic Facial Expression Recognition

Martínez, Brais; Valstar, Michel

doi:10.1007/978-3-319-25958-1_4

Cited by 90 publications

(60 citation statements)

References 79 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Facial landmark detection plays a crucial role in a number of research areas and applications such as facial attribute detection [18], facial expression analysis [22], emotion recognition and sentiment analysis [43,41,23,39], and 3D facial reconstruction [14]. A full review of work in facial landmark detection is outside the scope of this paper and we refer the reader to recent reviews of the field [11,37].…”

Section: Related Workmentioning

confidence: 99%

Convolutional Experts Constrained Local Model for Facial Landmark Detection

Zadeh

Baltrušaitis

Morency

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Constrained Local Models (CLMs) are a well-established family of methods for facial landmark detection. However, they have recently fallen out of favor to cascaded regression-based approaches. This is in part due to the inability of existing CLM local detectors to model the very complex individual landmark appearance that is affected by expression, illumination, facial hair, makeup, and accessories. In our work, we present a novel local detector -Convolutional Experts Network (CEN) -that brings together the advantages of neural architectures and mixtures of experts in an end-to-end framework. We further propose a Convolutional Experts Constrained Local Model (CE-CLM) algorithm that uses CEN as local detectors. We demonstrate that our proposed CE-CLM algorithm outperforms competitive state-of-the-art baselines for facial landmark detection by a large margin on four publicly-available datasets. Our approach is especially accurate and robust on challenging profile images.

show abstract

Section: Related Workmentioning

confidence: 99%

Convolutional Experts Constrained Local Model for Facial Landmark Detection

Zadeh

Baltrušaitis

Morency

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

show abstract

“…Although the affect model based on basic emotions is limited in the ability to represent the complexity and subtlety of our daily affective displays [7], [8], [9], and other emotion description models, such as the Facial Action Coding System (FACS) [10] and the continuous model using affect dimensions [11], are considered to represent a wider range of emotions, the categorical model that describes emotions in terms of discrete basic emotions is still the most popular perspective for FER, due to its pioneering investigations along with the direct and intuitive definition of facial expressions. And in this survey, we will limit our discussion on FER based on the categorical model.…”

Section: Introductionmentioning

confidence: 99%

Deep Facial Expression Recognition: A Survey

Deng

2022

IEEE Trans. Affective Comput.

1,015

593

View full text Add to dashboard Cite

With the transition of facial expression recognition (FER) from laboratory-controlled to challenging in-the-wild conditions and the recent success of deep learning techniques in various fields, deep neural networks have increasingly been leveraged to learn discriminative representations for automatic FER. Recent deep FER systems generally focus on two important issues: overfitting caused by a lack of sufficient training data and expression-unrelated variations, such as illumination, head pose and identity bias. In this paper, we provide a comprehensive survey on deep FER, including datasets and algorithms that provide insights into these intrinsic problems. First, we introduce the available datasets that are widely used in the literature and provide accepted data selection and evaluation principles for these datasets. We then describe the standard pipeline of a deep FER system with the related background knowledge and suggestions of applicable implementations for each stage. For the state of the art in deep FER, we review existing novel deep neural networks and related training strategies that are designed for FER based on both static images and dynamic image sequences, and discuss their advantages and limitations. Competitive performances on widely used benchmarks are also summarized in this section. We then extend our survey to additional related issues and application scenarios. Finally, we review the remaining challenges and corresponding opportunities in this field as well as future directions for the design of robust deep FER systems.

show abstract

“…Similarly, [43] includes different data modalities, different affect models and historical considerations on the topic. Other works providing an overview include [35] and [107], which focus primarily on applications and problems related to facial AUs, and [50], which provides a more in-depth explanation of a sub-set of methods rather than a general overview. This work provides a comprehensive survey of recent efforts in the field and focuses exclusively on automatic AU analysis from RGB imagery.…”

Section: Introductionmentioning

confidence: 99%

Automatic Analysis of Facial Actions: A Survey

Martínez

Valstar

Jiang

et al. 2019

IEEE Trans. Affective Comput.

Self Cite

224

153

View full text Add to dashboard Cite

Abstract-As one of the most comprehensive and objective ways to describe facial expressions, the Facial Action Coding System (FACS) has recently received significant attention. Over the past 30 years, extensive research has been conducted by psychologists and neuroscientists on various aspects of facial expression analysis using FACS. Automating FACS coding would make this research faster and more widely applicable, opening up new avenues to understanding how we communicate through facial expressions. Such an automated process can also potentially increase the reliability, precision and temporal resolution of coding. This paper provides a comprehensive survey of research into machine analysis of facial actions. We systematically review all components of such systems: pre-processing, feature extraction and machine coding of facial actions. In addition, the existing FACS-coded facial expression databases are summarised. Finally, challenges that have to be addressed to make automatic facial action analysis applicable in real-life situations are extensively discussed. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the future of machine recognition of facial actions: what are the challenges and opportunities that researchers in the field face.

show abstract

Advances, Challenges, and Opportunities in Automatic Facial Expression Recognition

Cited by 90 publications

References 79 publications

Convolutional Experts Constrained Local Model for Facial Landmark Detection

Convolutional Experts Constrained Local Model for Facial Landmark Detection

Deep Facial Expression Recognition: A Survey

Automatic Analysis of Facial Actions: A Survey

Contact Info

Product

Resources

About