Emotion Expression from Different Angles: A Video Database for Facial Expressions of Actors Shot by a Camera Array

Seuß, Dominik; Dieckmann, Anja; Hassan, Teena; Garbas, Jens-Uwe; Ellgring, Johann Heinrich; Mortillaro, Marcello; Scherer, Klaus R.

doi:10.1109/acii.2019.8925458

Cited by 16 publications

(17 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It also produced temporally smooth AU intensity estimates, which facilitates the identification of the different temporal phases of AUs. In general, the estimates from the proposed approach are conservative, with delayed onsets and lower range of values for estimated showing the effect of one-time versus continuous facial shape calibration on a sequence from the Actor Study Database [168], in which the actor displays AU43-EyesClosed. One-time shape calibration was performed iteratively on the first frame which was duplicated for 20 time steps.…”

Section: Performance Evaluationmentioning

confidence: 98%

“…Qualitative analysis of the outputs from the two systems was also performed. For the evaluation, the Actor Study Database [168], the UNBC-McMaster Shoulder Pain Expression Archive Database [121], and a proprietary marketresearch database [70] were used. The results, along with a discussion of the pros and cons of the two systems, are presented in detail in Preprint C.2.1.…”

Section: Automatic Facial Action Estimationmentioning

confidence: 99%

“…The results, along with a discussion of the pros and cons of the two systems, are presented in detail in Preprint C.2.1. Benchmark results were computed on the Actor Study Database [168] using the system developed by Dapogny et al [30], and are included in Publication B.2.3.…”

Section: Automatic Facial Action Estimationmentioning

confidence: 99%

“…This is due to the reduced interference from SU parameters, as they are estimated separately. Although the qualitative results speak in favour of one-time facial shape calibration, this approach requires a neutral (a) One-time shape calibration (b) Continuous shape calibration Figure 3.14: Qualitative results showing the positions of facial landmarks estimated using one-time versus continuous facial shape calibration for the first frame of a sequence from the Actor Study Database [168]. In the one-time approach, only the SU and rigid parameters are used to fit the deformable facial shape model to the observed facial landmark detections.…”

Section: Adapting To Person-dependent Variationsmentioning

confidence: 99%

“…In addition to these, the performance of the AU intensity estimation approach was compared with the performance of a state-of-the-art system for AU recognition [30]. The evaluation was performed on three datasets, namely the Actor Study Database [168], the UNBC-McMaster Shoulder Pain Expression Archive Database [121], and a proprietary market-research database that is not yet published. All datasets contain AU annotations at frame-level.…”

Section: Performance Evaluationmentioning

confidence: 99%

See 4 more Smart Citations

Towards Robust and Interpretable Practical Applications of Automatic Mental State Analysis Using a Dynamic and Hybrid Facial Action Estimation Approach

Hassan¹

Self Cite

View full text Add to dashboard Cite

This dissertation presents a probabilistic state estimation framework for integrating data-driven machine learning models and a deformable facial shape model in order to estimate continuous-valued intensities of 22 different facial muscle movements, known as Action Units (AU), defined in the Facial Action Coding System (FACS). A practical approach is proposed and validated for integrating class-wise probability scores from machine learning models within a Gaussian state estimation framework. Furthermore, driven mass-spring-damper models are applied for modelling the dynamics of facial muscle movements. Both facial shape and appearance information are used for estimating AU intensities, making it a hybrid approach. Several features are designed and explored to help the probabilistic framework to deal with multiple challenges involved in automatic AU detection. The proposed AU intensity estimation method and its features are evaluated quantitatively and qualitatively using three different datasets containing either spontaneous or acted facial expressions with AU annotations. The proposed method produced temporally smoother estimates that facilitate a fine-grained analysis of facial expressions. It also performed reasonably well, even though it simultaneously estimates intensities of 22 AUs, some of which are subtle in expression or resemble each other closely. The estimated AU intensities tended to the lower range of values, and were often accompanied by a small delay in onset. This shows that the proposed method is conservative. In order to further improve performance, state-of-the-art machine learning approaches for AU detection could be integrated within the proposed probabilistic AU intensity estimation framework. In addition to AU intensity estimation, this dissertation explores the applicability of the estimated AU intensities for automatic analysis of mental states such as pain and distraction. A survey of automatic pain detection approaches, conducted as part of this dissertation, highlights the progress and deficits in this field. Several AU-based rules are designed for pain intensity estimation based on psychological evidence, and their performance is evaluated empirically. The potential of these AU-based rules to automatically generate explanations for pain detections is also illustrated. Furthermore, a preliminary analysis of the estimated AU intensities shows differences in facial actions between various distraction scenarios during simulated driving. The results of this dissertation show that more interdisciplinary research is needed to address the open challenges in the field of automatic mental state analysis, particularly to build reference datasets, to model interpersonal differences, and to generate human-comprehensible explanations of predictions.

show abstract

Section: Performance Evaluationmentioning

confidence: 98%

Section: Automatic Facial Action Estimationmentioning

confidence: 99%

Section: Automatic Facial Action Estimationmentioning

confidence: 99%

Section: Adapting To Person-dependent Variationsmentioning

confidence: 99%

Section: Performance Evaluationmentioning

confidence: 99%

See 3 more Smart Citations

Towards Robust and Interpretable Practical Applications of Automatic Mental State Analysis Using a Dynamic and Hybrid Facial Action Estimation Approach

Hassan¹

Self Cite

View full text Add to dashboard Cite

show abstract

Emotional Meta-annotation of Educational Videos. A Review of Emotional Database Characterization

Astudillo¹,

Sanz²,

Baldassarri³

2022

Communications in Computer and Information Science

View full text Add to dashboard Cite

Domain-Specific Evaluation of Visual Explanations for Application-Grounded Facial Expression Recognition

Finzel,

Rieger,

Kuhn

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Research in the field of explainable artificial intelligence has produced a vast amount of visual explanation methods for deep learning-based image classification in various domains of application. However, there is still a lack of domain-specific evaluation methods to assess an explanation’s quality and a classifier’s performance with respect to domain-specific requirements. In particular, evaluation methods could benefit from integrating human expertise into quality criteria and metrics. Such domain-specific evaluation methods can help to assess the robustness of deep learning models more precisely. In this paper, we present an approach for domain-specific evaluation of visual explanation methods in order to enhance the transparency of deep learning models and estimate their robustness accordingly. As an example use case, we apply our framework to facial expression recognition. We can show that the domain-specific evaluation is especially beneficial for challenging use cases such as facial expression recognition and provides application-grounded quality criteria that are not covered by standard evaluation methods. Our comparison of the domain-specific evaluation method with standard approaches thus shows that the quality of the expert knowledge is of great importance for assessing a model’s performance precisely.

show abstract

Emotion Expression from Different Angles: A Video Database for Facial Expressions of Actors Shot by a Camera Array

Cited by 16 publications

References 25 publications

Towards Robust and Interpretable Practical Applications of Automatic Mental State Analysis Using a Dynamic and Hybrid Facial Action Estimation Approach

Towards Robust and Interpretable Practical Applications of Automatic Mental State Analysis Using a Dynamic and Hybrid Facial Action Estimation Approach

Emotional Meta-annotation of Educational Videos. A Review of Emotional Database Characterization

Domain-Specific Evaluation of Visual Explanations for Application-Grounded Facial Expression Recognition

Contact Info

Product

Resources

About