Investigating the Use of Formant Based Features for Detection of Affective Dimensions in Speech

Kim, Jonathan C.; Rao, Hrishikesh; Clements, Mark A.

doi:10.1007/978-3-642-24571-8_48

Cited by 19 publications

(9 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the median duration of the turns in the SEMAINE corpus is 2.76 secs, the delay is significant and the resulting labels do not represent the actual expressive behaviors. We hypothesize that this is one of the reasons of the low emotion recognition performance reported in classification studies on this database [33], [34], [35].…”

Section: Related Workmentioning

confidence: 92%

Correcting Time-Continuous Emotional Labels by Modeling the Reaction Lag of Evaluators

Mariooryad

Busso

2015

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

An appealing scheme to characterize expressive behaviors is the use of emotional dimensions such as activation (calm versus active) and valence (negative versus positive). These descriptors offer many advantages to describe the wide spectrum of emotions. Due to the continuous nature of fast-changing expressive vocal and gestural behaviors, it is desirable to continuously track these emotional traces, capturing subtle and localized events (e.g., with FEELTRACE). However, time-continuous annotations introduce challenges that affect the reliability of the labels. In particular, an important issue is the evaluators' reaction lag caused by observing, appraising, and responding to the expressive behaviors. An empirical analysis demonstrates that this delay varies from 1 to 6 seconds, depending on the annotator, expressive dimension, and actual behaviors. Our experiments show accuracy improvements even with fixed delays (1-3 seconds). This paper proposes to compensate for this reaction lag by finding the time-shift that maximizes the mutual information between the expressive behaviors and the time-continuous annotations. The approach is implemented by making different assumptions about the evaluators' reaction lag. The benefits of compensating for the delay is demonstrated with emotion classification experiments. On average, the classifiers trained with facial and speech features show more than 7 percent relative improvements over baseline classifiers trained and tested without shifting the time-continuous annotations.

show abstract

Section: Related Workmentioning

confidence: 92%

Correcting Time-Continuous Emotional Labels by Modeling the Reaction Lag of Evaluators

Mariooryad

Busso

2015

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…Here we compare our audio only approach against the baselines as well as three state-of-the-art approaches [3,5,2]. The results are shown in Table 1.…”

Section: Audio Sub-challengementioning

confidence: 97%

“…There has been extensive work on human emotion recognition in recent years [2,3,4,5]. Recognizing that human emotion varies dynamically, several works have used techniques such as HMMs [3] and CRFs (and its variations) [2] for analyzing human emotions.…”

Section: Introductionmentioning

confidence: 99%

Affect analysis in natural human interaction using Joint Hidden Conditional Random Fields

Siddiquie

Khan

Divakaran

et al. 2013

2013 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

We present a novel approach for multi-modal affect analysis in human interactions that is capable of integrating data from multiple modalities while also taking into account temporal dynamics. Our fusion approach, Joint Hidden Conditional Random Fields (JHRCFs), combines the advantages of purely feature level (early fusion) fusion approaches with late fusion (CRFs on individual modalities) to simultaneously learn the correlations between features from multiple modalities as well as their temporal dynamics. Our approach addresses major shortcomings of other fusion approaches such as the domination of other modalities by a single modality with early fusion and the loss of cross-modal information with late fusion. Extensive results on the AVEC 2011 dataset show that we outperform the state-of-the-art on the Audio Sub-Challenge, while achieving competitive performance on the Video SubChallenge and the Audiovisual Sub-Challenge.

show abstract

“…For AVEC 2011: UCL (Meng and Bianchi-Berthouze 2011), Uni-ULM (Glodek et al 2011), GaTechKim (Kim et al 2011), LSU (Calix et al 2011), Waterloo (Sayedelahl et al 2011), NLPR (Pan et al 2011), USC (Ramirez et al 2011), GaTechSun (Sun and Moore 2011), I2R-SCUT (Cen et al 2011), UCR (Cruz et al 2011) and UMontreal (Dahmane and Meunier 2011a, b). For AVEC 2012: UPMC-UAG (Nicolle et al 2012), Supelec-Dynamixyz-MinesTelecom (Soladie et al 2012), UPenn (Savran et al 2012a), USC (Ozkan et al 2012), Delft (van der Maaten 2012), Uni-ULM (Glodek et al 2012), Waterloo2 (Fewzee and Karray 2012).…”

Section: Audio/visual Emotion Challenge 2011/2012mentioning

confidence: 99%

Automatic Facial Expression Analysis

Valstar

2014

Understanding Facial Expressions in Communication

View full text Add to dashboard Cite

Investigating the Use of Formant Based Features for Detection of Affective Dimensions in Speech

Cited by 19 publications

References 6 publications

Correcting Time-Continuous Emotional Labels by Modeling the Reaction Lag of Evaluators

Correcting Time-Continuous Emotional Labels by Modeling the Reaction Lag of Evaluators

Affect analysis in natural human interaction using Joint Hidden Conditional Random Fields

Automatic Facial Expression Analysis

Contact Info

Product

Resources

About