Enhancing Multilingual Recognition of Emotion in Speech by Language Identification

Sagha, Hesam; Matějka, Pavel; Gavryukova, Maryna; Povolny, Filip; Marchi, Erik; Schuller, Björn

doi:10.21437/interspeech.2016-333

Cited by 26 publications

(16 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the existing approaches are tailored toward specific databases, which could be one of the main factors making this task tough to solve. While the system is trained on a particular database, it faces the issues of different subjects, their ethnicity, appearance, culture, sex and age [31], contextual meaning of sentences, and the background noise [32]. Consequently, the algorithm does not work well when dealing with natural environment [33].…”

Section: Introductionmentioning

confidence: 99%

Audiovisual emotion recognition in wild

Avots

Sapiński

Bachmann

et al. 2018

Machine Vision and Applications

128

View full text Add to dashboard Cite

People express emotions through different modalities. Utilization of both verbal and nonverbal communication channels allows to create a system in which the emotional state is expressed more clearly and therefore easier to understand. Expanding the focus to several expression forms can facilitate research on emotion recognition as well as human-machine interaction. This article presents analysis of audiovisual information to recognize human emotions. A cross-corpus evaluation is done using three different databases as the training set (SAVEE, eNTERFACE'05 and RML) and AFEW (database simulating realworld conditions) as a testing set. Emotional speech is represented by commonly known audio and spectral features as well as MFCC coefficients. The SVM algorithm has been used for classification. In case of facial expression, faces in key frames are found using Viola-Jones face recognition algorithm and facial image emotion classification done by CNN (AlexNet). Multimodal emotion recognition is based on decision-level fusion. The performance of emotion recognition algorithm is compared with the validation of human decision makers.

show abstract

Section: Introductionmentioning

confidence: 99%

Audiovisual emotion recognition in wild

Avots

Sapiński

Bachmann

et al. 2018

Machine Vision and Applications

128

View full text Add to dashboard Cite

show abstract

“…An approach to multilingual emotion classification using language identification and model selection is presented in [7]. In contrast to this work where language-dependent models are trained and then selected accordingly, we examine the performance of one model trained on multiple languages.…”

Section: Introductionmentioning

confidence: 99%

CRoss-lingual and Multilingual Speech Emotion Recognition on English and French

Neumann

2018

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Research on multilingual speech emotion recognition faces the problem that most available speech corpora differ from each other in important ways, such as annotation methods or interaction scenarios. These inconsistencies complicate building a multilingual system. We present results for crosslingual and multilingual emotion recognition on English and French speech data with similar characteristics in terms of interaction (human-human conversations). Further, we explore the possibility of fine-tuning a pre-trained cross-lingual model with only a small number of samples from the target language, which is of great interest for low-resource languages. To gain more insights in what is learned by the deployed convolutional neural network, we perform an analysis on the attention mechanism inside the network.

show abstract

“…Moreover, speech linguistic features carry information about the culture and the way emotions are expressed or perceived. In [5], a model selection technique based on language identification is proposed to improve speech emotion recognition accuracy. As per the census, tribal people make up about 8.2 percent of the India's total population.…”

Section: Motivationmentioning

confidence: 99%

Design and Development of a Multi-Lingual Speech Corpora (TaMaR-EmoDB) for Emotion Analysis

Rajan¹,

Haritha²,

C³

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

This paper presents the design, the development of a new multilingual emotional speech corpus, TaMaR-EmoDB (Tamil Malayalam Ravula-Emotion DataBase) and its evaluation using a deep neural network (DNN)-baseline system. The corpus consists of utterances from three languages, namely, Malayalam, Tamil and Ravula, a tribal language. The database consists of short speech utterances in four emotions-anger, anxiety, happiness, and sadness, along with neutral utterances. The subset of the corpus is first evaluated using a perception test, in order to understand how well the emotional state in emotional speech is identified by humans. Later, machine testing is performed using the fusion of spectral and prosodic features with DNN framework. During the classification phase, the system reports an average precision of 0.78, 0.60, 0.61 and recall of 0.84, 0.61 and 0.53 for Malayalam, Tamil, and Ravula, respectively. This database can potentially be used as a new linguistic resource that will enable future research in speech emotion detection, corpus-based prosody analysis, and speech synthesis.

show abstract

Enhancing Multilingual Recognition of Emotion in Speech by Language Identification

Cited by 26 publications

References 19 publications

Audiovisual emotion recognition in wild

Audiovisual emotion recognition in wild

CRoss-lingual and Multilingual Speech Emotion Recognition on English and French

Design and Development of a Multi-Lingual Speech Corpora (TaMaR-EmoDB) for Emotion Analysis

Contact Info

Product

Resources

About