Towards Speech Emotion Recognition "in the wild" using Aggregated Corpora and Deep Multi-Task Learning

Kim, Jaebok; Englebienne, Gwenn; Truong, Khiet P.; Evers, Vanessa

doi:10.48550/arxiv.1708.03920

Cited by 9 publications

(16 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comparison UA with other state-of-the-art methods on the EMO-DB corpus. The DANN method [6] and AL method [7] results are taken from the re-implementation in paper [5]. Therefore, we cite those results from [5] directly.…”

Section: Resultsmentioning

confidence: 99%

“…Our UCNNPM largely outperforms the few-shot based method with the UA increase of 45.5%. Since the code for domain adversarial neural network (DANN) [6] and the aggregated multi-task learning (AL) [7] are not publicly available. We cite these results on EMO-DB from re-implementation in the paper [5].…”

Section: Methodsmentioning

confidence: 99%

“…Consistent with the data from the previous databases, four emotional categories (joy (happy), sadness, anger, and neutral) were selected for use in the experiment. Following the practive in [7], audios from actors #15 and # 16 were used as test partition and validation partition respectively while audios from the rest of the actor were used as the training partition.…”

Section: Germanmentioning

confidence: 99%

“…Unsupervised methods that do not need data labels in the target domain provide an alternative solution for cross-lingual SER, especially for low-resource languages. Some crosslingual SER systems based on unsupervised learning methods are introduced recently [4,5,6,7]. A mainstream method is to utilize the adversarial-based method since it can learn the corpus-invariant representations using domain adversarial training.…”

Section: Introductionmentioning

confidence: 99%

“…However, the effectiveness of domain adversarial training is highly correlated with the distribution of two databases so that adversarial attacks and instabilities may occur during training if the data points are sharply different from each other [9]. Multi-task learning training methodology for unsupervised cross-lingual is another approach to improve the generalisability of the model by incorporating information on gender and naturalness [7]. An alternative apporach is proposed by training to learn the emotion similarities from source domain samples through few-shot learning adapted to the target domain [5].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Unsupervised Cross-Lingual Speech Emotion Recognition Using Pseudo Multilabel

Li¹,

Yan²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

Speech Emotion Recognition (SER) in a single language has achieved remarkable results through deep learning approaches in the last decade. However, cross-lingual SER remains a challenge in real-world applications due to a great difference between the source and target domain distributions. To address this issue, we propose an Unsupervised Cross-Lingual Neural Network with Pseudo Multilabel (UC-NNPM) that is trained to learn the emotion similarities between source domain features inside an external memory adjusted to identify emotion in cross-lingual databases. UC-NNPM introduces a novel approach that leverages external memory to store source domain features and generates pseudo multilabel for each target domain data by computing the similarities between the external memory and the target domain features. We evaluate our approach on multiple different languages of speech emotion databases. Experimental results show our proposed approach significantly improves the weighted accuracy (WA) across multiple low-resource languages on Urdu, Skropus, ShEMO, and EMO-DB corpus.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Germanmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Unsupervised Cross-Lingual Speech Emotion Recognition Using Pseudo Multilabel

Li¹,

Yan²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

show abstract

Disentanglement for Audio-Visual Emotion Recognition Using Multitask Setup

Peri

Parthasarathy

Bradshaw

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Deep learning models trained on audio-visual data have been successfully used to achieve state-of-the-art performance for emotion recognition. In particular, models trained with multitask learning have shown additional performance improvements. However, such multitask models entangle information between the tasks, encoding the mutual dependencies present in label distributions in the real world data used for training. This work explores the disentanglement of multimodal signal representations for the primary task of emotion recognition and a secondary person identification task. In particular, we developed a multitask framework to extract low-dimensional embeddings that aim to capture emotion specific information, while containing minimal information related to person identity. We evaluate three different techniques for disentanglement and report results of up to 13% disentanglement while maintaining emotion recognition performance.

show abstract

Improving Cross-Corpus Speech Emotion Recognition with Adversarial Discriminative Domain Generalization (ADDoG)

Gideon

McInnis

Provost

2021

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Automatic speech emotion recognition provides computers with critical context to enable user understanding. While methods trained and tested within the same dataset have been shown successful, they often fail when applied to unseen datasets. To address this, recent work has focused on adversarial methods to find more generalized representations of emotional speech. However, many of these methods have issues converging, and only involve datasets collected in laboratory conditions. In this paper, we introduce Adversarial Discriminative Domain Generalization (ADDoG), which follows an easier to train "meet in the middle" approach. The model iteratively moves representations learned for each dataset closer to one another, improving cross-dataset generalization. We also introduce Multiclass ADDoG, or MADDoG, which is able to extend the proposed method to more than two datasets, simultaneously. Our results show consistent convergence for the introduced methods, with significantly improved results when not using labels from the target dataset. We also show how, in most cases, ADDoG and MADDoG can be used to improve upon baseline state-of-the-art methods when target dataset labels are added and in-the-wild data are considered. Even though our experiments focus on cross-corpus speech emotion, these methods could be used to remove unwanted factors of variation in other settings. , and a co-author of the winner of the Classifier Sub-Challenge event at the Interspeech 2009 emotion challenge. Her research interests are in human-centered speech and video processing, multimodal interfaces design, and speech-based assistive technology. The goals of her research are motivated by the complexities of the perception and expression of human behavior.

show abstract

Towards Speech Emotion Recognition "in the wild" using Aggregated Corpora and Deep Multi-Task Learning

Cited by 9 publications

References 17 publications

Unsupervised Cross-Lingual Speech Emotion Recognition Using Pseudo Multilabel

Unsupervised Cross-Lingual Speech Emotion Recognition Using Pseudo Multilabel

Disentanglement for Audio-Visual Emotion Recognition Using Multitask Setup

Improving Cross-Corpus Speech Emotion Recognition with Adversarial Discriminative Domain Generalization (ADDoG)

Contact Info

Product

Resources

About