What Should a Generic Emotion Markup Language Be Able to Represent?

Schröder, Marc; Devillers, Laurence; Karpouzis, Kostas; Martin, Jean‐Claude; Pélachaud, Catherine; Peter, Christian; Pirker, Hannes; Schuller, Björn W.; Tao, Jianhua; Wilson, Ian D.

doi:10.1007/978-3-540-74889-2_39

Cited by 33 publications

(15 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The HUMAINE EARL [170] can be named as one of the first standards tailored to provide a well-defined description of recognised or to be synthesised emotions. Released as a working draft, the W3C EmotionML 9 [170,169] followed EARL providing more flexibility and a broader coverage including action tendencies, appraisals, meta context or a basis to encode regulation, acting, meta-data, and ontologies. Particularly for the encoding of acoustic and linguistic features, a standard was proposed and used within the CEICES initiative [24].…”

Section: Implementation and System Integrationmentioning

confidence: 99%

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Schuller

Batliner

Steidl

et al. 2011

Speech Communication

Self Cite

607

329

View full text Add to dashboard Cite

More than a decade has passed since research on automatic recognition of emotion from speech has become a new field of research in line with its 'big brothers' speech and speaker recognition. This article attempts to provide a short overview on where we are today, how we got there and what this can reveal us on where to go next and how we could arrive there. In a first part, we address the basic phenomenon reflecting the last fifteen years, commenting on databases, modelling and annotation, the unit of analysis and prototypicality. We then shift to automatic processing including discussions on features, classification, robustness, evaluation, and implementation and system integration. From there we go to the first comparative challenge on emotion recognition from speech -the INTERSPEECH 2009 Emotion Challenge, organised by (part of) the authors, including the description of the Challenge's database, Sub-Challenges, participants and their approaches, the winners, and the fusion of results to the actual learnt lessons before we finally address the ever-lasting problems and future promising attempts.Keywords: emotion, affect, automatic classification, feature types, feature selection, noise robustness, adaptation, standardisation, usability, evaluation Setting the SceneThis special issue will address new approaches towards dealing with the processing of realistic emotions in speech, and this overview article will give an account of the state-of-the-art, of the lacunas in this field, and of promising approaches towards overcoming shortcomings in modelling and recognising realistic emotions. We will also report on the first emotion challenge at INTERSPEECH 2009, constituting the initial impetus of this special issue; to end with, we want to sketch future strategies and applications, trying to answer the question 'Where to go from here?'The article is structured as follows: we first deal with the basic phenomenon briefly reflecting the last fifteen years, commenting on databases, modelling and annotation, the unit of analysis and prototypicality. We then proceed to automatic processing (sec. 2) including discussions on features, classification, robustness, evaluation, and implementation and system integration. From there we go to the the first Emotion Challenge (sec. 3) including the description of the Challenge's database, Sub-Challenges, participants and their approaches, the winners, and the fusion of results to the lessons learnt, before concluding this article (sec. 4).

show abstract

Section: Implementation and System Integrationmentioning

confidence: 99%

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Schuller

Batliner

Steidl

et al. 2011

Speech Communication

Self Cite

607

329

View full text Add to dashboard Cite

show abstract

“…For our encoding scheme, we decided in favour of a straightforward ASCII representation: one line for each extracted feature; each column is attributed a unique semantics. This encoding can be easily converted into a Markup Language such as the one envisaged by Schröder et al (2007), cf. as well Schröder et al (2006).…”

Section: Featuresmentioning

confidence: 99%

Whodunnit – Searching for the most important feature types signalling emotion-related user states in speech

Batliner

Steidl

Schuller

et al. 2011

Computer Speech & Language

Self Cite

127

View full text Add to dashboard Cite

In this article, we describe and interpret a set of acoustic and linguistic features that characterise emotional/emotion-related user states -confined to the one database processed: four classes in a German corpus of children interacting with a pet robot. To this end, we collected a very large feature vector consisting of more than 4000 features extracted at different sites. We performed extensive feature selection (Sequential Forward Floating Search) for seven acoustic and four linguistic types of features, ending up in a small number of 'most important' features which we try to interpret by discussing the impact of different feature and extraction types. We establish different measures of impact and discuss the mutual influence of acoustics and linguistics.

show abstract

“…It can be seen that the rough organisation follows the simple tripartition of input (left), central processing (middle), and output (right), and that arrows indicate a rough pipeline for the data flow, from input analysis via central processing to output generation. In particular for the emotion coding, EmotionML is used [26] which already allows for continuous spatiotemporal emotion representation.…”

Section: The Challenge Of Continuous Emotion Recognitionmentioning

confidence: 99%

On-line emotion recognition in a 3-D activation-valence-time continuum using acoustic and linguistic cues

Eyben

Wöllmer

Graves

et al. 2009

J Multimodal User Interfaces

Self Cite

128

View full text Add to dashboard Cite

For many applications of emotion recognition, such as virtual agents, the system must select responses while the user is speaking. This requires reliable on-line recognition of the user's affect. However most emotion recognition systems are based on turnwise processing. We present a novel approach to on-line emotion recognition from speech using Long Short-Term Memory Recurrent Neural Networks. Emotion is recognised frame-wise in a two-dimensional valence-activation continuum. In contrast to current state-of-the-art approaches, recognition is performed on low-level signal frames, similar to those used for speech recognition. No statistical functionals are applied to low-level feature contours. Framing at a higher level is therefore unnecessary and regression outputs can be produced in real-time for every low-level input frame. We also investigate the benefits of including linguistic features on the signal frame level obtained by a keyword spotter.

show abstract

What Should a Generic Emotion Markup Language Be Able to Represent?

Cited by 33 publications

References 8 publications

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Whodunnit – Searching for the most important feature types signalling emotion-related user states in speech

On-line emotion recognition in a 3-D activation-valence-time continuum using acoustic and linguistic cues

Contact Info

Product

Resources

About