Deep-Net: A Lightweight CNN-Based Speech Emotion Recognition System Using Deep Frequency Features

Abstract

Artificial intelligence (AI) and machine learning (ML) are employed to make systems smarter. Today, the speech emotion recognition (SER) system evaluates the emotional state of the speaker by investigating his/her speech signal. Emotion recognition is a challenging task for a machine. In addition, making it smarter so that the emotions are efficiently recognized by AI is equally challenging. The speech signal is quite hard to examine using signal processing methods because it consists of different frequencies and features that vary according to emotions, such as anger, fear, sadness, happiness, boredom, disgust, and surprise. Even though different algorithms are being developed for the SER, the success rates are very low according to the languages, the emotions, and the databases. In this paper, we propose a new lightweight effective SER model that has a low computational complexity and a high recognition accuracy. The suggested method uses the convolutional neural network (CNN) approach to learn the deep frequency features by using a plain rectangular filter with a modified pooling strategy that have more discriminative power for the SER. The proposed CNN model was trained on the extracted frequency features from the speech data and was then tested to predict the emotions. The proposed SER model was evaluated over two benchmarks, which included the interactive emotional dyadic motion capture (IEMOCAP) and the berlin emotional speech database (EMO-DB) speech datasets, and it obtained 77.01% and 92.02% recognition results. The experimental results demonstrated that the proposed CNN-based SER system can achieve a better recognition performance than the state-of-the-art SER systems.

On the Speech Properties and Feature Extraction Methods in Speech Emotion Recognition

“…In contrary to that, there are few articles testing and analyzing the behavior of specific features and their settings in given conditions, e.g., testing frequency ranges or scales [ 44 ], etc. Nevertheless, published results measured even on the same database vary a lot, e.g., from approximately 50% [ 45 ] to even 92% [ 46 ], mainly due to the experimental set up, evaluation, processing, and classification. This study differs as it provides a unified, complex, and statistically rigorous analysis of great variety of basic speech properties, features and their settings, and calculation methods related to SER, by means of the machine learning.…”

A Novel Approach for Classification of Speech Emotions Based on Deep and Acoustic Features

“…The method presented in this study consists of acoustic features, deep features, pre-trained CNN and SVM combined model. In many studies, acoustic and deep features are used separately [11], [12], [16], [17]. In this study, acoustic and deep features are combined to improve the semantic information of the emotion features in the speech.…”

“…Generally, the essential feature parameters utilized in the speech emotion recognition system can be separated into two categories in terms of conventional features and deep features. Features extracted from Convolutional Neural Network (CNN) layers are generally used as deep features [11], [12]. In [13], to recognize emotions from speech, a method that is based on MFCC features and Gaussian mixture model classifier is proposed.…”

Multi-Path and Group-Loss-Based Network for Speech Emotion Recognition in Multi-Domain Datasets

“…Recent SER models based on deep-learning architectures [ 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 ] have demonstrated state-of-the-art performance with an attention mechanism [ 19 , 20 , 22 , 23 , 25 , 26 ]. The deep-learning architectures adopted in previous studies included recurrent neural networks (RNN) [ 19 ], convolutional neural networks (CNN) [ 24 ], and convolutional RNNs (CRNN) [ 20 , 26 ]. Liu et al [ 21 ] presented an SER model of a decision tree for an extreme learning machine having a single hidden-layer feed-forward neural network, using a mixture of deep learning and typical classification techniques.…”