Emotion Recognition in Naturalistic Speech and Language—A Survey

Weninger, Felix; Wöllmer, Martin; Schuller, Björn

doi:10.1002/9781118910566.ch10

Cited by 26 publications

(19 citation statements)

References 99 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Across different modalities, acoustic features were shown to perform the best. Despite these few notable exceptions, the existing work on automatic emotion recognition from facial and body responses mainly focus on the recognition of few emotions with prototypical expressions (Calvo and D'Mello, 2010;Weninger et al, 2015). Recognition of emotions using continuous dimensions, such as valence and arousal, has been also explored but these attempts did not include interest (Hatice et al, 2011;Sariyanidi et al, 2015).…”

Section: Detectionmentioning

confidence: 99%

Behavioral and Physiological Responses to Visual Interest and Appraisals: Multimodal Analysis and Automatic Recognition

Soleymani

Mortillaro

2018

Front. ICT

View full text Add to dashboard Cite

Interest drives our focus of attention and plays an important role in social communication. Given its relevance for many activities (e.g., learning, entertainment) a system able to automatically detect someone's interest has several potential applications. In this paper, we analyze the physiological and behavioral patterns associated with visual interest and present a method for the automatic recognition of interest, curiosity and their most relevant appraisals, namely, coping potential, novelty and complexity. We conducted an experiment in which participants watched images and micro-videos while multimodal signals were recorded-facial expressions, galvanic skin response (GSR), and eye gaze. After watching each stimulus, participants self-reported their level of interest, curiosity, coping potential, perceived novelty, and complexity. Results showed that interest was associated with other facial Action Units than smiling when dynamics was taken into consideration, especially inner brow raiser and eye lid tightener. Longer saccades were also present when participants watched interesting stimuli. However, correlations of appraisals with specific facial Action Units and eye gaze were in general stronger than those we found for interest. We trained Random Forests regression models to detect the level of interest, curiosity, and appraisals from multimodal features. The recognition models-unimodal and multimodal-for appraisals generally outperformed those for interest, in particular for static images. In summary, our study suggests that automatic appraisal detection may be a suitable way to detect subtle emotions like interest for which prototypical expressions do not exist.

show abstract

Section: Detectionmentioning

confidence: 99%

Behavioral and Physiological Responses to Visual Interest and Appraisals: Multimodal Analysis and Automatic Recognition

Soleymani

Mortillaro

2018

Front. ICT

View full text Add to dashboard Cite

show abstract

“…As outlined above, a number of information-bearing signals and methods exist to analyse users' affective state, including most notably speech emotion recognition [31], opinion and sentiment analysis from text or also multiple modalities, or the interpretation of visual signals, motion capture, and thermal imaging signals [11], and bio signals which are best (all) combined in a synergistic modality fusion. A number of different classification schemes are available to obtain best results depending on the application context and can be optimised in the loop with an according performance evaluation.…”

Section: Automatic Analysis and Predictionmentioning

confidence: 99%

Modelling User Affect and Sentiment in Intelligent User Interfaces

Schuller

2015

Proceedings of the 20th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

The computer-based automatic analysis of human sentiment, and affect are broadly expected to play a major role that will likely make 'that difference' in future Intelligent User Interfaces, as they bear the promise to lend interactive systems emotional intelligence. Such comprise intelligent digital games, e. g., for empowerment and inclusion, tutoring systems, information systems or virtual companions, e. g., in the car to name but a few. This tutorial aims to give a good introduction into the related fields of user Sentiment Analysis and user Affect Modelling. Its intention is to show the general technology, and its current reliability, the ways for technical integration and efficient embedding of solutions in a user interface context, and the latest trends in this young and ever emerging field. Emphasis is laid on highlighting the range of toolkits available at this moment with the aim of empowering one to immediately craft own solutions. This description contains the general motivation,

show abstract

“…In literature, several approaches for automatic emotion recognition are focused on the variety of human interaction capabilities or biological data. For instance, the study of speech and other acoustic cues in [36], body movements in [5], electroencephalogram (EEG) in [17], facial expressions or combinations of previous ones, such as speech and facial expressions in [21] or EEG and facial expressions in [30]. The study of facial expression has been part of various disciplines since Aristotelian era but it was only in 1978 when the first automatic recognition study appeared [27,2].…”

Section: Introductionmentioning

confidence: 99%

Understanding Human Activities Through 3D Sensors

2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Cognitive behaviour analysis is considered of high importance with many innovative applications in a range of sectors including healthcare, education, robotics and entertainment. In healthcare, cognitive and emotional behaviour analysis helps to improve the quality of life of patients and their families. Amongst all the di↵erent approaches for cognitive behaviour analysis, significant work has been focused on emotion analysis through facial expressions using depth and EEG data. Our work introduces an emotion recognition approach using facial expressions based on depth data and landmarks. A novel dataset was created that triggers emotions from long or short term memories. This work uses novel features based on a non-linear dimensionality reduction, t-SNE, applied on facial landmarks and depth data. Its performance was evaluated in a comparative study, proving that our approach outperforms other state-of-the-art features.

show abstract

Emotion Recognition in Naturalistic Speech and Language—A Survey

Cited by 26 publications

References 99 publications

Behavioral and Physiological Responses to Visual Interest and Appraisals: Multimodal Analysis and Automatic Recognition

Behavioral and Physiological Responses to Visual Interest and Appraisals: Multimodal Analysis and Automatic Recognition

Modelling User Affect and Sentiment in Intelligent User Interfaces

Understanding Human Activities Through 3D Sensors

Contact Info

Product

Resources

About