High-level feature representation using recurrent neural network for speech emotion recognition

Lee, Jin-Kyu; Tashev, Ivan

doi:10.21437/interspeech.2015-336

Cited by 271 publications

(146 citation statements)

References 15 publications

Supporting

Mentioning

144

Contrasting

Unclassified

Order By: Relevance

“…The Angry and Neutral categories exhibit much lower performance than the Happy and Sad categories, which can be understood from the presented scatter plots in Figure 2, as the Angry class has an overlap with Happy and Neutral categories, and Neutral has an overlap with all the other categories. (Lee & Tashev, 2015) report an unweighted and weighted accuracy of 52.13%, and 57.91% respectively for the DNN-ELM model.…”

Section: Frame Based Representations With the Denoising Autoencodermentioning

confidence: 94%

“…(1) A network with the same architecture without pre-training (2) A softmax classifier trained on features extracted from the COVAREP toolbox, such as MFCCs and prosodic features (for example, pitch, peak slope, Normalized Amplitude Quotient (NAQ), and difference between first two harmonics in speech (H1-H2)) and (3) the DNN-ELM approach described in Han et al (2014), for which results in the same experimental setting (emotion categories and speaker-independent data splits) as our work have been reported in Lee & Tashev (2015). The Angry and Neutral categories exhibit much lower performance than the Happy and Sad categories, which can be understood from the presented scatter plots in Figure 2, as the Angry class has an overlap with Happy and Neutral categories, and Neutral has an overlap with all the other categories.…”

Section: Frame Based Representations With the Denoising Autoencodermentioning

confidence: 99%

See 1 more Smart Citation

Learning Representations of Affect from Speech

Ghosh,

Laksana,

Morency

et al. 2015

Preprint

View full text Add to dashboard Cite

There has been a lot of prior work on representation learning for speech recognition applications, but not much emphasis has been given to an investigation of effective representations of affect from speech, where the paralinguistic elements of speech are separated out from the verbal content. In this paper, we explore denoising autoencoders for learning paralinguistic attributes, i.e. categorical and dimensional affective traits from speech. We show that the representations learnt by the bottleneck layer of the autoencoder are highly discriminative of activation intensity and at separating out negative valence (sadness and anger) from positive valence (happiness). We experiment with different input speech features (such as FFT and log-mel spectrograms with temporal context windows), and different autoencoder architectures (such as stacked and deep autoencoders). We also learn utterance specific representations by a combination of denoising autoencoders and BLSTM based recurrent autoencoders. Emotion classification is performed with the learnt temporal/dynamic representations to evaluate the quality of the representations. Experiments on a well-established real-life speech dataset (IEMO-CAP) show that the learnt representations are comparable to state of the art feature extractors (such as voice quality features and MFCCs) and are competitive with state-of-the-art approaches at emotion and dimensional affect recognition.

show abstract

Section: Frame Based Representations With the Denoising Autoencodermentioning

confidence: 94%

Section: Frame Based Representations With the Denoising Autoencodermentioning

confidence: 99%

Learning Representations of Affect from Speech

Ghosh,

Laksana,

Morency

et al. 2015

Preprint

View full text Add to dashboard Cite

show abstract

“…Mao et al [3] firstly introduced Convolutional Neural Networks (CNNs) for the SER task and obtained remarkable results on various datasets by learning affective-salient features. Recurrent neural networks (RNNs) has also been introduced for SER purpose with a deep Bidirectional Long Short-Term Memory (BLTSM) network proposed by Lee et al [4]. Several papers have then presented CNNs in combination with LSTM cells to improve speech emotion recognition, based on log Mel filterbanks (logMel) [5] or raw signal in an end-to-end manner [6].…”

Section: Related Workmentioning

confidence: 99%

Speaker Attentive Speech Emotion Recognition

Moine¹,

Obin²,

Röebel³

2021

Preprint

View full text Add to dashboard Cite

Speech Emotion Recognition (SER) task has known significant improvements over the last years with the advent of Deep Neural Networks (DNNs). However, even the most successful methods are still rather failing when adaptation to specific speakers and scenarios is needed, inevitably leading to poorer performances when compared to humans. In this paper, we present novel work based on the idea of teaching the emotion recognition network about speaker identity. Our system is a combination of two ACRNN classifiers respectively dedicated to speaker and emotion recognition. The first informs the latter through a Self Speaker Attention (SSA) mechanism that is shown to considerably help to focus on emotional information of the speech signal. Experiments on social attitudes database Att-HACK and IEMOCAP corpus demonstrate the effectiveness of the proposed method and achieve the state-of-the-art performance in terms of unweighted average recall.

show abstract

“…The total number of speakers in the corpus is 10. We only considered the samples belonging to the four emotional categories of happiness, sadness, neutral and anger, to keep the analysis consistent with previous works [6,7,8,9,10,11,15,16]. The number of utterances in each emotional class of each speaker is shown in Table 1.…”

Section: Corpusmentioning

confidence: 99%

“…Initial DNN-based models [4] were still based on the same utterance-level feature extraction. However, in subsequent approaches, speech features extracted from each frame were used as inputs of more complex neural network architectures such as Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), and the accuracy was further improved [5,6,7]. Recent years saw the application of novel methods developed from other AI fields, such as self-attention models [8], Connectionist Temporal Classification (CTC) [9] and Dilated Residual Network (DRN) [10].…”

Section: Introductionmentioning

confidence: 99%

Addressing Ambiguity of Emotion Labels Through Meta-Learning

Fujioka¹,

Bertero²,

Homma³

et al. 2019

Preprint

View full text Add to dashboard Cite

Emotion labels in emotion recognition corpora are highly noisy and ambiguous, due to the annotators' subjective perception of emotions. Such ambiguity may introduce errors in automatic classification and affect the overall performance. We therefore propose a dynamic label correction and sample contribution weight estimation model. Our model is based on a standard BLSTM model with attention with two extra parameters. The first learns a new corrected label distribution, and is aimed to fix the inaccurate labels from the dataset. The other instead estimates the contribution of each sample to the training process, and is aimed to ignore the ambiguous and noisy samples while giving higher weight to the clear ones. We train our model through an alternating optimization method, where in the first epoch we update the neural network parameters, and in the second we keep them fixed to update the label correction and sample importance parameters. When training and evaluating our model on the IEMOCAP dataset, we obtained a weighted accuracy (WA) and unweighted accuracy (UA) of respectively 65.9% and 61.4%. This yielded an absolute improvement of 2.5%, 2.7% respectively compared to a BLSTM with attention baseline, trained on the corpus gold labels.

show abstract

High-level feature representation using recurrent neural network for speech emotion recognition

Cited by 271 publications

References 15 publications

Learning Representations of Affect from Speech

Learning Representations of Affect from Speech

Speaker Attentive Speech Emotion Recognition

Addressing Ambiguity of Emotion Labels Through Meta-Learning

Contact Info

Product

Resources

About