Cross lingual speech emotion recognition using canonical correlation analysis on principal component subspace

Sagha, Hesam; Deng, Jun; Gavryukova, Maryna; Schuller, Björn

doi:10.1109/icassp.2016.7472789

Cited by 38 publications

(37 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Building upon this work, Mao et al [22] proposed to learn a shared feature representation across domains by constraining their model to share the class priors across domains. Sagha et al [23], also motivated by the work of Deng et al [11], used principal component analysis (PCA) along with kernel canonical correlation analysis (KCCA) to find views with the highest correlation between the source and target corpora. First, they used PCA to represent the feature space of the source and target data.…”

Section: Related Workmentioning

confidence: 99%

Domain Adversarial for Acoustic Emotion Recognition

Abdelwahab

Busso

2018

IEEE/ACM Trans. Audio Speech Lang. Process.

150

View full text Add to dashboard Cite

The performance of speech emotion recognition is affected by the differences in data distributions between train (source domain) and test (target domain) sets used to build and evaluate the models. This is a common problem, as multiple studies have shown that the performance of emotional classifiers drop when they are exposed to data that does not match the distribution used to build the emotion classifiers. The difference in data distributions becomes very clear when the training and testing data come from different domains, causing a large performance gap between validation and testing performance. Due to the high cost of annotating new data and the abundance of unlabeled data, it is crucial to extract as much useful information as possible from the available unlabeled data. This study looks into the use of adversarial multitask training to extract a common representation between train and test domains. The primary task is to predict emotional attribute-based descriptors for arousal, valence, or dominance. The secondary task is to learn a common representation where the train and test domains cannot be distinguished. By using a gradient reversal layer, the gradients coming from the domain classifier are used to bring the source and target domain representations closer. We show that exploiting unlabeled data consistently leads to better emotion recognition performance across all emotional dimensions. We visualize the effect of adversarial training on the feature representation across the proposed deep learning architecture. The analysis shows that the data representations for the train and test domains converge as the data is passed to deeper layers of the network. We also evaluate the difference in performance when we use a shallow neural network versus a deep neural network (DNN) and the effect of the number of shared layers used by the task and domain classifiers.Index Terms-Speech emotion recognition, adversarial training, unlabeled adaptation of acoustic emotional models.

show abstract

Section: Related Workmentioning

confidence: 99%

Domain Adversarial for Acoustic Emotion Recognition

Abdelwahab

Busso

2018

IEEE/ACM Trans. Audio Speech Lang. Process.

150

View full text Add to dashboard Cite

show abstract

“…Interestingly, approaches in the AVEC 2018 CES did not employ approaches such as transfer learning [80,81] or domain adaptation techniques [29,54] typically seen in cross-cultural testing. In [76], the authors proposed a model based on emotional salient detection to identify emotion markers invariant to sociocultural context.…”

Section: Cross-cultural Emotion Recognitionmentioning

confidence: 99%

AVEC 2018 Workshop and Challenge

Ringeval

Schuller

Valstar

et al. 2018

Proceedings of the 2018 on Audio/Visual Emotion Challenge and Workshop

120

View full text Add to dashboard Cite

The Audio/Visual Emotion Challenge and Workshop (AVEC 2019) "State-of-Mind, Detecting Depression with AI, and Cross-cultural Affect Recognition" is the ninth competition event aimed at the comparison of multimedia processing and machine learning methods for automatic audiovisual health and emotion analysis, with all participants competing strictly under the same conditions. The goal of the Challenge is to provide a common benchmark test set for multimodal information processing and to bring together the health and emotion recognition communities, as well as the audiovisual processing communities, to compare the relative merits of various approaches to health and emotion recognition from reallife data. This paper presents the major novelties introduced this year, the challenge guidelines, the data used, and the performance of the baseline systems on the three proposed tasks: state-of-mind *

show abstract

“…Successful baseline experiments evaluating the automatic emotion recognition component have been performed in earlier work [3,7,18] and were herein expanded by performing novel BoAW experiments, leading to a UAR of 43.3% (4 emotion classes) on the collected data and promising a large margin for improvement considering recent machine learning techniques such as generative models and transfer learning. Furthermore, primary usability evaluations with the target group are currently ongoing, giving first insights into the promising success of the system.…”

Section: Discussionmentioning

confidence: 99%

Introducing an Emotion-Driven Assistance System for Cognitively Impaired Individuals

Hantke

Cohrs

Schmitt

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Mental, neurological and/or physical disabilities often affect individuals' cognitive processes, which in turn can introduce difficulties with remembering what they have learnt. Therefore, completing trivial daily tasks can be challenging and supervision or help from others is constantly needed. In this regard, these individuals with special needs can benefit from nowadays advanced assistance techniques. Within this contribution, a language-driven, workplace integrated, assistance system is being proposed, supporting disabled individuals in the handling of certain activities while taking into account their emotional-cognitive constitution and state. In this context, we present a set of baseline results for emotion recognition tasks and conduct machine learning experiments to benchmark the performance of an automatic emotion recognition system on the collected data. We show that this is a challenging task that can nevertheless be tackled with state-of-the-art methodologies.

show abstract

Cross lingual speech emotion recognition using canonical correlation analysis on principal component subspace

Cited by 38 publications

References 18 publications

Domain Adversarial for Acoustic Emotion Recognition

Domain Adversarial for Acoustic Emotion Recognition

AVEC 2018 Workshop and Challenge

Introducing an Emotion-Driven Assistance System for Cognitively Impaired Individuals

Contact Info

Product

Resources

About