Unsupervised Features for Facial Expression Intensity Estimation Over Time

Awiszus, Maren; GraBhof, Stella; Kuhnke, Felix; Östermann, Jörn

doi:10.1109/cvprw.2018.00156

Cited by 5 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Automatic quantitative assessment of facial expression can be defined as the ability of a system to analyze facial cues and to give a numerical outcome describing how much the produced facial expression is similar to the expected one in an objective way. The lack of standard rules for expression intensity labeling and the limited availability of labeled data are the two most drawbacks in this research area that includes only a few recent pioneering works limited to a single expression [ 14 ], making use of 3D data [ 15 ] or requiring unrealistic constrained evolutions such as an initial neutral stage (with no expression) followed by the onset of expression ending with expression apex [ 16 ].…”

Section: Introductionmentioning

confidence: 99%

Computational Assessment of Facial Expression Production in ASD Children

Leo

Carcagnì

Distante

et al. 2018

Sensors

View full text Add to dashboard Cite

In this paper, a computational approach is proposed and put into practice to assess the capability of children having had diagnosed Autism Spectrum Disorders (ASD) to produce facial expressions. The proposed approach is based on computer vision components working on sequence of images acquired by an off-the-shelf camera in unconstrained conditions. Action unit intensities are estimated by analyzing local appearance and then both temporal and geometrical relationships, learned by Convolutional Neural Networks, are exploited to regularize gathered estimates. To cope with stereotyped movements and to highlight even subtle voluntary movements of facial muscles, a personalized and contextual statistical modeling of non-emotional face is formulated and used as a reference. Experimental results demonstrate how the proposed pipeline can improve the analysis of facial expressions produced by ASD children. A comparison of system’s outputs with the evaluations performed by psychologists, on the same group of ASD children, makes evident how the performed quantitative analysis of children’s abilities helps to go beyond the traditional qualitative ASD assessment/diagnosis protocols, whose outcomes are affected by human limitations in observing and understanding multi-cues behaviors such as facial expressions.

show abstract

Section: Introductionmentioning

confidence: 99%

Computational Assessment of Facial Expression Production in ASD Children

Leo

Carcagnì

Distante

et al. 2018

Sensors

View full text Add to dashboard Cite

show abstract

“…Our method is free from data labeling cost by being an unsupervised feature learning approach. Note that there exist a few number of unsupervised approaches in the same and/or related topics, e.g., speech emotion recognition [30], [31], facial emotion recognition [32], facial expression intensity estimation [33], and multimodal sentiment and emotion analysis [25]. However, our method involves the deep architectures either pre-trained on tasks different from emotion recognition (e.g., action recognition) or not pre-trained.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Emotion Recognition with Modality-Pairwise Unsupervised Contrastive Loss

Franceschini¹,

Fini²,

Beyan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Emotion recognition is involved in several real-world applications. With an increase in available modalities, automatic understanding of emotions is being performed more accurately. The success in Multimodal Emotion Recognition (MER), primarily relies on the supervised learning paradigm. However, data annotation is expensive, time-consuming, and as emotion expression and perception depends on several factors (e.g., age, gender, culture) obtaining labels with a high reliability is hard. Motivated by these, we focus on unsupervised feature learning for MER. We consider discrete emotions, and as modalities text, audio and vision are used. Our method, as being based on contrastive loss between pairwise modalities, is the first attempt in MER literature. Our end-to-end feature learning approach has several differences (and advantages) compared to existing MER methods: i) it is unsupervised, so the learning is lack of data labelling cost; ii) it does not require data spatial augmentation, modality alignment, large number of batch size or epochs; iii) it applies data fusion only at inference; and iv) it does not require backbones pre-trained on emotion recognition task. The experiments on benchmark datasets show that our method outperforms several baseline approaches and unsupervised learning methods applied in MER. Particularly, it even surpasses a few supervised MER state-of-the-art.

show abstract