Interpreting ambiguous emotional expressions

Mower, Emily; Metallinou, Angeliki; Lee, Chi-Chun; Kazemzadeh, Abe; Busso, Carlos; Lee, Sungbok; Narayanan, Shrikanth

doi:10.1109/acii.2009.5349500

Cited by 113 publications

(60 citation statements)

References 26 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although ambiguous, many patterns proved to be still important and do characterise spontaneous emotional speech. Similar results obtained on different datasets and with different classifiers are confirmed by [132] where models created for prototypical emotional speech proved to be uneffective for the classification of non-prototypical utterances.…”

Section: Prototypicality Vs Open Microphone Settingsupporting

confidence: 71%

“…This is of course a more realistic setting, mandatory for real-life applications, but yielding lower classification performance [132,193,213,211].…”

Section: Prototypicality Vs Open Microphone Settingmentioning

confidence: 99%

“…[20,202,132,209,213]. If we do not select but deal with the whole database, classes might be 'noisy' and classification performance rather low.…”

Section: Prototypicality Vs Open Microphone Settingmentioning

confidence: 99%

See 2 more Smart Citations

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Schuller

Batliner

Steidl

et al. 2011

Speech Communication

595

329

View full text Add to dashboard Cite

More than a decade has passed since research on automatic recognition of emotion from speech has become a new field of research in line with its 'big brothers' speech and speaker recognition. This article attempts to provide a short overview on where we are today, how we got there and what this can reveal us on where to go next and how we could arrive there. In a first part, we address the basic phenomenon reflecting the last fifteen years, commenting on databases, modelling and annotation, the unit of analysis and prototypicality. We then shift to automatic processing including discussions on features, classification, robustness, evaluation, and implementation and system integration. From there we go to the first comparative challenge on emotion recognition from speech -the INTERSPEECH 2009 Emotion Challenge, organised by (part of) the authors, including the description of the Challenge's database, Sub-Challenges, participants and their approaches, the winners, and the fusion of results to the actual learnt lessons before we finally address the ever-lasting problems and future promising attempts.Keywords: emotion, affect, automatic classification, feature types, feature selection, noise robustness, adaptation, standardisation, usability, evaluation Setting the SceneThis special issue will address new approaches towards dealing with the processing of realistic emotions in speech, and this overview article will give an account of the state-of-the-art, of the lacunas in this field, and of promising approaches towards overcoming shortcomings in modelling and recognising realistic emotions. We will also report on the first emotion challenge at INTERSPEECH 2009, constituting the initial impetus of this special issue; to end with, we want to sketch future strategies and applications, trying to answer the question 'Where to go from here?'The article is structured as follows: we first deal with the basic phenomenon briefly reflecting the last fifteen years, commenting on databases, modelling and annotation, the unit of analysis and prototypicality. We then proceed to automatic processing (sec. 2) including discussions on features, classification, robustness, evaluation, and implementation and system integration. From there we go to the the first Emotion Challenge (sec. 3) including the description of the Challenge's database, Sub-Challenges, participants and their approaches, the winners, and the fusion of results to the lessons learnt, before concluding this article (sec. 4).

show abstract

Section: Prototypicality Vs Open Microphone Settingsupporting

confidence: 71%

“…This is of course a more realistic setting, mandatory for real-life applications, but yielding lower classification performance [132,193,213,211].…”

Section: Prototypicality Vs Open Microphone Settingmentioning

confidence: 99%

See 1 more Smart Citation

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Schuller

Batliner

Steidl

et al. 2011

Speech Communication

595

329

View full text Add to dashboard Cite

show abstract

“…Mower et al [7] argued that it was very unlikely that each spoken utterance during natural human robot/computer interaction contained clear emotional content. Thus, dialog modeling techniques, such as emotional interpolation, have been developed in their work to interpret those emotionally ambiguous or nonprototypical utterances.…”

Section: Related Workmentioning

confidence: 99%

Affect Detection from Text-Based Virtual Improvisation and Emotional Gesture Recognition

Zhang

Yap

2012

Advances in Human-Computer Interaction

View full text Add to dashboard Cite

We have developed an intelligent agent to engage with users in virtual drama improvisation previously. The intelligent agent was able to perform sentence-level affect detection from user inputs with strong emotional indicators. However, we noticed that many inputs with weak or no affect indicators also contain emotional implication but were regarded as neutral expressions by the previous interpretation. In this paper, we employ latent semantic analysis to perform topic theme detection and identify target audiences for such inputs. We also discuss how such semantic interpretation of the dialog contexts is used to interpret affect more appropriately during virtual improvisation. Also, in order to build a reliable affect analyser, it is important to detect and combine weak affect indicators from other channels such as body language. Such emotional body language detection also provides a nonintrusive channel to detect users' experience without interfering with the primary task. Thus, we also make initial exploration on affect detection from several universally accepted emotional gestures.

show abstract

“…To fulfill this research gap, Mower et al [16,17] proposed a feature-agglomerate extraction method to encompass all appeared distinctive emotions in single prediction. Their approach coincides with the foregoing ML-ARAM model in ensuring the structured multi-label predictions.…”

Section: Previous Work On Affective Learningmentioning

confidence: 99%

Uncertainty Flow Facilitates Zero-Shot Multi-Label Learning in Affective Facial Analysis

2018

View full text Add to dashboard Cite

Featured Application: The proposed Uncertainty Flow framework may benefit the facial analysis with its promised elevation in discriminability in multi-label affective classification tasks. Moreover, this framework also allows the efficient model training and between tasks knowledge transfer. The applications that rely heavily on continuous prediction on emotional valance, e.g., to monitor prisoners' emotional stability in jail, can be directly benefited from our framework. Abstract:To lower the single-label dependency on affective facial analysis, it urges the fruition of multi-label affective learning. The impediment to practical implementation of existing multi-label algorithms pertains to scarcity of scalable multi-label training datasets. To resolve this, an inductive transfer learning based framework, i.e., Uncertainty Flow, is put forward in this research to allow knowledge transfer from a single labelled emotion recognition task to a multi-label affective recognition task. I.e., the model uncertainty-which can be quantified in Uncertainty Flow-is distilled from a single-label learning task. The distilled model uncertainty ensures the later efficient zero-shot multi-label affective learning. On the theoretical perspective, within our proposed Uncertainty Flow framework, the feasibility of applying weakly informative priors, e.g., uniform and Cauchy prior, is fully explored in this research. More importantly, based on the derived weight uncertainty, three sets of prediction related uncertainty indexes, i.e., soft-max uncertainty, pure uncertainty and uncertainty plus are proposed to produce reliable and accurate multi-label predictions. Validated on our manual annotated evaluation dataset, i.e., the multi-label annotated FER2013, our proposed Uncertainty Flow in multi-label facial expression analysis exhibited superiority to conventional multi-label learning algorithms and multi-label compatible neural networks. The success of our proposed Uncertainty Flow provides a glimpse of future in continuous, uncertain, and multi-label affective computing.

show abstract

Interpreting ambiguous emotional expressions

Cited by 113 publications

References 26 publications

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Affect Detection from Text-Based Virtual Improvisation and Emotional Gesture Recognition

Uncertainty Flow Facilitates Zero-Shot Multi-Label Learning in Affective Facial Analysis

Contact Info

Product

Resources

About