Cross-language acoustic emotion recognition: An overview and some tendencies

Feraru, Silvia Monica; Schuller, Dagmar; Schuller, Bjoern

doi:10.1109/acii.2015.7344561

Cited by 52 publications

(34 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The training emotion corpora are either (i) the other language within the same language family (diagonals), (ii) the aggregation of the corpora in the other language family (off-diagonals), or (iii) both within and across language families ('All' column). As indicated in [11], we expect to have high accuracies on diagonals (i. e., same language family) with respect to the offdiagonals (i. e., cross language family). This is valid for the Sino-Tibetan and Romance families.…”

Section: Resultsmentioning

confidence: 95%

“…Likewise, Scherer et al concluded that cultureand language-specific paralinguistic patterns may influence the emotion perception [10]. Furthermore, Feraru et al investigated emotion recognition from speech on cross-language families by including less researched languages from completly different language families such as Burmese, Romanian or Turkish [11]. They found that, AER for corpora of the same language has the highest accuracy while emotion recognition across language families has the lowest.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Multilingual Recognition of Emotion in Speech by Language Identification

et al. 2016

View full text Add to dashboard Cite

We investigate, for the first time, if applying model selection based on automatic language identification (LID) can improve multilingual recognition of emotion in speech. Six emotional speech corpora from three language families (Germanic, Romance, Sino-Tibetan) are evaluated. The emotions are represented by the quadrants in the arousal/valence plane, i. e., positive/negative arousal/valence. Four selection approaches for choosing an optimal training set depending on the current language are compared: within the same language family, across language family, use of all available corpora, and selection based on the automatic LID. We found that, on average, the proposed LID approach for selecting training corpora is superior to using all the available corpora when the spoken language is not known.

show abstract

Section: Resultsmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

Enhancing Multilingual Recognition of Emotion in Speech by Language Identification

et al. 2016

View full text Add to dashboard Cite

show abstract

“…Among them, Variety encompasses multimodality (audio, video, text) and multilinguality/multiculturalism, and Veracity emerges from subjectivity of assessments (annotations). These aspects have been addressed for: (i) the textual modality by: automatic translation [17], defining multilingual WordNet Grid [18], and (ii) for the audio modality by: analyzing within or between language family emotion recognition [19], feature transfer learning between languages [20], model transfer learning [21], language identification [22], audio denoising [23], and decision aggregation through cooperative speaker models [24]. Regarding the Volume and Velocity, there is a need for fast computation.…”

Section: Emotion Analysis In Big Data and Pre-requisitesmentioning

confidence: 99%

“…NIF also provides different URI Schemes to identify text fragments inside a string, e. g., a scheme based on RFC5147 [91], and a custom scheme based on context. To this end, texts are converted to RDF 19 literals and a URI 20 is generated so that linked data annotations can be defined for that text. The same idea can also be applied to annotate multimedia [92].…”

Section: Linked Data and Knowledge Graphmentioning

confidence: 99%

MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis

Buitelaar

Wood

Negi

et al. 2018

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Recently, there is an increasing tendency to embed functionalities for recognizing emotions from user generated media content in automated systems such as call-centre operations, recommendations and assistive technologies, providing richer and more informative user and content profiles. However, to date, adding these functionalities was a tedious, costly, and time consuming effort, requiring identification and integration of diverse tools with diverse interfaces as required by the use case at hand. The MixedEmotions Toolbox leverages the need for such functionalities by providing tools for text, audio, video, and linked data processing within an easily integrable plug-and-play platform. These functionalities include: (i) for text processing: emotion and sentiment recognition, (ii) for audio processing: emotion, age, and gender recognition, (iii) for video processing: face detection and tracking, emotion recognition, facial landmark localization, head pose estimation, face alignment, and body pose estimation, and (iv) for linked data: knowledge graph integration. Moreover, the MixedEmotions Toolbox is open-source and free. In this article, we present this toolbox in the context of the existing landscape, and provide a range of detailed benchmarks on standard test-beds showing its state-of-the-art performance. Furthermore, three real-world use-cases show its effectiveness, namely emotion-driven smart TV, call center monitoring, and brand reputation analysis.

show abstract

“…Speech-based affective computing is now well developed. There is nearly thirty years of research related to speech-based affective computing [1] and 66% of the world's native language speaking populations are represented by affective speech data sets [2]. Despite a large body of evidence linking eye and head-based cues to emotion and motivational state conveyance [3]- [12], the use of these cues is underdeveloped for affective computing purposes.…”

Section: Introductionmentioning

confidence: 99%

Speech, Head, and Eye-based Cues for Continuous Affect Prediction

O'Dwyer

2019

2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)

View full text Add to dashboard Cite

Continuous affect prediction involves the discrete time-continuous regression of affect dimensions. Dimensions to be predicted often include arousal and valence. Continuous affect prediction researchers are now embracing multimodal model input. This provides motivation for researchers to investigate previously unexplored affective cues. Speech-based cues have traditionally received the most attention for affect prediction, however, non-verbal inputs have significant potential to increase the performance of affective computing systems and in addition, allow affect modelling in the absence of speech. However, nonverbal inputs that have received little attention for continuous affect prediction include eye and head-based cues. The eyes are involved in emotion displays and perception while headbased cues have been shown to contribute to emotion conveyance and perception. Additionally, these cues can be estimated noninvasively from video, using modern computer vision tools. This work exploits this gap by comprehensively investigating head and eye-based features and their combination with speech for continuous affect prediction. Hand-crafted, automatically generated and CNN-learned features from these modalities will be investigated for continuous affect prediction. The highest performing feature sets and feature set combinations will answer how effective these features are for the prediction of an individuals affective state.

show abstract

Cross-language acoustic emotion recognition: An overview and some tendencies

Cited by 52 publications

References 42 publications

Enhancing Multilingual Recognition of Emotion in Speech by Language Identification

Enhancing Multilingual Recognition of Emotion in Speech by Language Identification

MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis

Speech, Head, and Eye-based Cues for Continuous Affect Prediction

Contact Info

Product

Resources

About