Naturalistic Affective Expression Classification by a Multi-stage Approach Based on Hidden Markov Models

Meng, Hongying; Bianchi-Berthouze, Nadia

doi:10.1007/978-3-642-24571-8_49

Cited by 37 publications

(21 citation statements)

References 20 publications

(9 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the median duration of the turns in the SEMAINE corpus is 2.76 secs, the delay is significant and the resulting labels do not represent the actual expressive behaviors. We hypothesize that this is one of the reasons of the low emotion recognition performance reported in classification studies on this database [33], [34], [35].…”

Section: Related Workmentioning

confidence: 92%

Correcting Time-Continuous Emotional Labels by Modeling the Reaction Lag of Evaluators

Mariooryad

Busso

2015

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

An appealing scheme to characterize expressive behaviors is the use of emotional dimensions such as activation (calm versus active) and valence (negative versus positive). These descriptors offer many advantages to describe the wide spectrum of emotions. Due to the continuous nature of fast-changing expressive vocal and gestural behaviors, it is desirable to continuously track these emotional traces, capturing subtle and localized events (e.g., with FEELTRACE). However, time-continuous annotations introduce challenges that affect the reliability of the labels. In particular, an important issue is the evaluators' reaction lag caused by observing, appraising, and responding to the expressive behaviors. An empirical analysis demonstrates that this delay varies from 1 to 6 seconds, depending on the annotator, expressive dimension, and actual behaviors. Our experiments show accuracy improvements even with fixed delays (1-3 seconds). This paper proposes to compensate for this reaction lag by finding the time-shift that maximizes the mutual information between the expressive behaviors and the time-continuous annotations. The approach is implemented by making different assumptions about the evaluators' reaction lag. The benefits of compensating for the delay is demonstrated with emotion classification experiments. On average, the classifiers trained with facial and speech features show more than 7 percent relative improvements over baseline classifiers trained and tested without shifting the time-continuous annotations.

show abstract

Section: Related Workmentioning

confidence: 92%

Correcting Time-Continuous Emotional Labels by Modeling the Reaction Lag of Evaluators

Mariooryad

Busso

2015

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…Several methods are based on contextdependent frameworks. For example, Meng et al [11] propose a system based on Hidden Markov Models. Wollmer et al [23] investigate a more advanced technique based on context modeling using Long Short-Term Memory neural networks.…”

Section: Introductionmentioning

confidence: 99%

Robust continuous prediction of human emotions using multiscale dynamic cues

Nicolle

Rapp

Bailly

et al. 2012

Proceedings of the 14th ACM International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Designing systems able to interact with humans in a natural manner is a complex and far from solved problem. A key aspect of natural interaction is the ability to understand and appropriately respond to human emotions. This paper details our response to the Audio/Visual Emotion Challenge (AVEC'12) whose goal is to continuously predict four affective signals describing human emotions (namely valence, arousal, expectancy and power). The proposed method uses log-magnitude Fourier spectra to extract multiscale dynamic descriptions of signals characterizing global and local face appearance as well as head movements and voice. We perform a kernel regression with very few representative samples selected via a supervised weighted-distance-based clustering, that leads to a high generalization power. For selecting features, we introduce a new correlation-based measure that takes into account a possible delay between the labels and the data and significantly increases robustness. We also propose a particularly fast regressor-level fusion framework to merge systems based on di↵erent modalities. Experiments have proven the e ciency of each key point of the proposed method and we obtain very promising results.

show abstract

“…or audio (speech) features. Recently, Cambridge University introduced the emotional computer [20] and the MIT (Massachusetts Institute of Technology) Mood Meter [21] From 2011, these have participated in several international emotion recognition challenges, such as AVEC (Audio/Visual Emotion Challenge) or MediaEval (Benchmarking Initiative for Multimedia Evaluation) [22,23].…”

Section: Related Workmentioning

confidence: 99%

Facial Expression Emotion Detection for Real-Time Embedded Systems

et al. 2018

Self Cite

View full text Add to dashboard Cite

Abstract:Recently, real-time facial expression recognition has attracted more and more research. In this study, an automatic facial expression real-time system was built and tested. Firstly, the system and model were designed and tested on a MATLAB environment followed by a MATLAB Simulink environment that is capable of recognizing continuous facial expressions in real-time with a rate of 1 frame per second and that is implemented on a desktop PC. They have been evaluated in a public dataset, and the experimental results were promising. The dataset and labels used in this study were made from videos, which were recorded twice from five participants while watching a video. Secondly, in order to implement in real-time at a faster frame rate, the facial expression recognition system was built on the field-programmable gate array (FPGA). The camera sensor used in this work was a Digilent VmodCAM -stereo camera module. The model was built on the Atlys TM Spartan-6 FPGA development board. It can continuously perform emotional state recognition in real-time at a frame rate of 30. A graphical user interface was designed to display the participant's video in real-time and two-dimensional predict labels of the emotion at the same time.

show abstract

Naturalistic Affective Expression Classification by a Multi-stage Approach Based on Hidden Markov Models

Cited by 37 publications

References 20 publications

Correcting Time-Continuous Emotional Labels by Modeling the Reaction Lag of Evaluators

Correcting Time-Continuous Emotional Labels by Modeling the Reaction Lag of Evaluators

Robust continuous prediction of human emotions using multiscale dynamic cues

Facial Expression Emotion Detection for Real-Time Embedded Systems

Contact Info

Product

Resources

About