Impact of Feature Selection Algorithm on Speech Emotion Recognition Using Deep Convolutional Neural Network

Abstract

Speech emotion recognition (SER) plays a significant role in human–machine interaction. Emotion recognition from speech and its precise classification is a challenging task because a machine is unable to understand its context. For an accurate emotion classification, emotionally relevant features must be extracted from the speech data. Traditionally, handcrafted features were used for emotional classification from speech signals; however, they are not efficient enough to accurately depict the emotional states of the speaker. In this study, the benefits of a deep convolutional neural network (DCNN) for SER are explored. For this purpose, a pretrained network is used to extract features from state-of-the-art speech emotional datasets. Subsequently, a correlation-based feature selection technique is applied to the extracted features to select the most appropriate and discriminative features for SER. For the classification of emotions, we utilize support vector machines, random forests, the k-nearest neighbors algorithm, and neural network classifiers. Experiments are performed for speaker-dependent and speaker-independent SER using four publicly available datasets: the Berlin Dataset of Emotional Speech (Emo-DB), Surrey Audio Visual Expressed Emotion (SAVEE), Interactive Emotional Dyadic Motion Capture (IEMOCAP), and the Ryerson Audio Visual Dataset of Emotional Speech and Song (RAVDESS). Our proposed method achieves an accuracy of 95.10% for Emo-DB, 82.10% for SAVEE, 83.80% for IEMOCAP, and 81.30% for RAVDESS, for speaker-dependent SER experiments. Moreover, our method yields the best results for speaker-independent SER with existing handcrafted features-based SER approaches.

Incorporating Interpersonal Synchronization Features for Automatic Emotion Recognition from Visual and Audio Data during Communication

“…The percentage of train-dataset is 70% and the number of emotion values is five. The main experiment is formulated as a speaker-dependent task, which is similar to [ 71 , 72 ].…”

Utterance Level Feature Aggregation with Deep Metric Learning for Speech Emotion Recognition

“…Within this context, the feature representation is used to train different machine learning systems, such as support vector machines [ 23 , 24 , 25 ], 1D-convolutional neural networks (1D-CNN) [ 26 , 27 ], 2D-CNN [ 28 , 29 , 30 , 31 , 32 ] or recurrent neural networks (RNN) [ 33 , 34 , 35 , 36 ].…”

“…In order to take advantage of the high robustness of deep convolutional neural networks, in [ 31 ], a pre-trained 2D-CNN is used in the context of speech emotion recognition. The CNN is designed to extract low-level features from state-of-the-art speech emotional datasets.…”

Hybrid Method of Automated EEG Signals’ Selection Using Reversed Correlation Algorithm for Improved Classification of Emotions

“…In the past decades, many researchers have investigated subjects’ reactions and potentials of different emotion recognition techniques, including speech, non-verbal audition, facial expression, visual and thermal images, peripheral neural signals, and central neural system signals [ 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 ].…”

“…The authors of [ 37 ] introduced a Deep Convolutional Neural Network (DCNN) during feature selection. They used several classification methods—support vector machine, random forest, the k-nearest neighbors’ algorithm, and neural network classifiers.…”