Autoencoder With Emotion Embedding for Speech Emotion Recognition

Abstract

An important part of the human-computer interaction process is speech emotion recognition (SER), which has been receiving more attention in recent years. However, although a wide diversity of methods has been proposed in SER, these approaches still cannot improve the performance. A key issue in the low performance of the SER system is how to effectively extract emotion-oriented features. In this paper, we propose a novel algorithm, an autoencoder with emotion embedding, to extract deep emotion features. Unlike many previous works, instance normalization, which is a common technique in the style transfer field, is introduced into our model rather than batch normalization. Furthermore, the emotion embedding path in our method can lead the autoencoder to efficiently learn a priori knowledge from the label. It can enable the model to distinguish which features are most related to human emotion. We concatenate the latent representation learned by the autoencoder and acoustic features obtained by the openSMILE toolkit. Finally, the concatenated feature vector is utilized for emotion classification. To improve the generalization of our method, a simple data augmentation approach is applied. Two publicly available and highly popular databases, IEMOCAP and EMODB, are chosen to evaluate our method. Experimental results demonstrate that the proposed model achieves significant performance improvement compared to other speech emotion recognition systems.

Emotion Recognition System Based on Two-Level Ensemble of Deep-Convolutional Neural Network Models

“…Zhang et al [61] proposed a novel methodology for recognizing emotions using speech. The authors used autoencoders along with emotion embedding to extract the deep emotion features.…”

Innovative Mechanism of Rural Finance: Risk Assessment Methods and Impact Factors of Agricultural Loans Based on Personal Emotion and Artificial Intelligence

“…In [ 21 ], the authors proposed a Focal Loss-based Convolutional Recurrent Neural Networks (FL-CRNN) deep learning model with variable input length for speech emotion recognition. In [ 22 ], the authors proposed an Automatic Encoder with Emotion Embedding (AEEE) to extract deep emotional features. In [ 23 ], Ozseven proposed a Statistical Feature Selection method based on the change of emotion on acoustic features (SFS-AF).…”

Intelligent Recognition and Analysis of Negative Emotions of Undergraduates Under COVID-19

“…Khattak et al ( 16 ) proposed an efficient deep learning technology, which used convolution neural networks to classify emotions from facial images and effectively detect age and gender from facial expressions. Zhang and Zhang ( 17 ) proposed an automatic encoder with emotion embedding to extract deep emotional features. Panahi et al ( 18 ) studied the effectiveness of fractional Fourier transform as a new feature extraction method in improving the accuracy of emotion recognition in physiological signals.…”