Emotion Recognition from Raw Speech using Wavenet

Pandey, Sandeep; Shekhawat, Hanumant Singh; Prasanna, S. R. Mahadeva

doi:10.1109/tencon.2019.8929257

Cited by 17 publications

(14 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The lexical features, however, labeled the sentences without any emotional words as "neutral", which biased the decision towards the "neutral" class for some cases, compensating the misclassification in the classifier based on acoustic features. Still, the performance for all four classes was rather similar in contrast to some of the reported papers [17,18,27,33], for which the differences between the highest and lowest detection probabilities among four classes range from 27.1% to 64%, resulting in high UARs.…”

Section: Discussionsupporting

confidence: 54%

“…Recently, deep learning approaches have been incorporated in speech emotion recognition and brought performance improvement [9][10][11][12][13][14][15][16][17][24][25][26][27][28][29][30][31][32][33]. In [24], a deep neural network (DNN) classifier was adopted for which the input consisted of acoustic features extracted from Mel frequency spectral coefficients using a convolutional neural network (CNN)-long short-term memory (LSTM), other acoustic features obtained by a DNN based on the LLD, and lexical features extracted as outputs of CNNs from words and parts-of-speech tags.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Affective Latent Representation of Acoustic and Lexical Features for Emotion Recognition

Kim¹,

Song

Shin

2020

Sensors

View full text Add to dashboard Cite

In this paper, we propose a novel emotion recognition method based on the underlying emotional characteristics extracted from a conditional adversarial auto-encoder (CAAE), in which both acoustic and lexical features are used as inputs. The acoustic features are generated by calculating statistical functionals of low-level descriptors and by a deep neural network (DNN). These acoustic features are concatenated with three types of lexical features extracted from the text, which are a sparse representation, a distributed representation, and an affective lexicon-based dimensions. Two-dimensional latent representations similar to vectors in the valence-arousal space are obtained by a CAAE, which can be directly mapped into the emotional classes without the need for a sophisticated classifier. In contrast to the previous attempt to a CAAE using only acoustic features, the proposed approach could enhance the performance of the emotion recognition because combined acoustic and lexical features provide enough discriminant power. Experimental results on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) corpus showed that our method outperformed the previously reported best results on the same corpus, achieving 76.72% in the unweighted average recall.

show abstract

Section: Discussionsupporting

confidence: 54%

Section: Introductionmentioning

confidence: 99%

Affective Latent Representation of Acoustic and Lexical Features for Emotion Recognition

Kim¹,

Song

Shin

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…MFCC considers human perception for sensitivity at appropriate frequencies by converting the conventional frequency to Mel scale; thus, it is suitable for speech recognition tasks. Alternatively, we use an audio generative model based on CNN, called WaveNet [24], [25], which is pre-trained based on NSynth dataset [26] to produce the salient representation of audio signals. The feature dimension of MFCC sequence is nT × C; where C is the number of coefficients, and feature dimension of WaveNet sequence is nT ×W .…”

Section: A Feature Extractions For Visual-audiomentioning

confidence: 99%

“…We determine feature representations for audio signals by computing the Mel-Frequency Cepstral Coefficients (MFCC) [20]- [23]. In addition, we extract other audio representations using an CNN-based generative model, called WaveNet [24], [25], which was pre-trained on NSynth [26]. After extracting all representations for visual and audio information, we feed these features to a deep graph fusion module to learn fused representations in a graph structure.…”

Section: Introductionmentioning

confidence: 99%

Deep Graph Fusion Based Multimodal Evoked Expressions From Large-Scale Videos

Yang

Kim

et al. 2021

IEEE Access

View full text Add to dashboard Cite

In the case of emotion recognition for wild input signals with large variances, multiple sources of noise can challenge the machine's ability to learn approximate ground truth. There are copious studies on recognizing characters' affective expressions directly through face, speech, and text. However, there are few researches on the prediction of a characters' emotions from their watched contents. Therefore, in this paper, we propose a hybrid fusion model, called deep graph fusion, to leverage the combination of visual-audio representations for predicting viewers' evoked expressions from videos. The proposed system includes four steps. First, we extract features for visual and audio modalities for each 30-second segment using CNN-based pre-trained models to learn their salient representations. Then, we reconstruct these features into graph outlines, and conduct node embedding using graph convolutional networks. In the third steps, we propose various fusion modules to combine the graph representations from visual and audio branches. Finally, the fused features are applied to Sigmoid activation in order to estimate the evoked scores for all emotional classes. Moreover, in order to enhance the overall performance, we propose a semantic embedding loss to learn the semantic meaning of textual emotions. We evaluate the proposed method using the Evoked Expression from Videos (EEV) database on both the validation and test sets. The experimental results demonstrate that the proposed algorithm outperforms all baseline models.

show abstract

“…In [9], a comparison of the effectiveness of traditional features versus end-to-end learning in atypical affect and crying recognition is presented, only to conclude that there is no clear winner. Moreover, works in [10] and [11] have also utilized raw speech for emotion classification.…”

Section: Introductionmentioning

confidence: 99%

Analysis of Emotional Content in Indian Political Speeches

Goel¹,

Pandey²,

Shekhawat³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Emotions play an essential role in public speaking. The emotional content of speech has the power to influence minds. As such, we present an analysis of the emotional content of politicians speech in the Indian political scenario. We investigate the emotional content present in the speeches of politicians using an Attention based CNN+LSTM network. Experimental evaluations on a dataset of eight Indian politicians shows how politicians incorporate emotions in their speeches to strike a chord with the masses. An analysis of the voting share received along with victory margin and their relation to emotional content in speech of the politicians is also presented.

show abstract

Emotion Recognition from Raw Speech using Wavenet

Cited by 17 publications

References 14 publications

Affective Latent Representation of Acoustic and Lexical Features for Emotion Recognition

Affective Latent Representation of Acoustic and Lexical Features for Emotion Recognition

Deep Graph Fusion Based Multimodal Evoked Expressions From Large-Scale Videos

Analysis of Emotional Content in Indian Political Speeches

Contact Info

Product

Resources

About