Recognition of speech emotion using custom 2D-convolution neural network deep learning algorithm

Abstract

Speech emotion recognition has become the heart of most human computer interaction applications in the modern world. The growing need to develop emotionally intelligent devices has opened up a lot of research opportunities. Most researchers in this field have applied the use of handcrafted features and machine learning techniques in recognising speech emotion. However, these techniques require extra processing steps and handcrafted features are usually not robust. They are computationally intensive because the curse of dimensionality results in low discriminating power. Research has shown that deep learning algorithms are effective for extracting robust and salient features in dataset. In this study, we have developed a custom 2D-convolution neural network that performs both feature extraction and classification of vocal utterances. The neural network has been evaluated against deep multilayer perceptron neural network and deep radial basis function neural network using the Berlin database of emotional speech, Ryerson audio-visual emotional speech database and Surrey audio-visual expressed emotion corpus. The described deep learning algorithm achieves the highest precision, recall and F1-scores when compared to other existing algorithms. It is observed that there may be need to develop customized solutions for different language settings depending on the area of applications.

Sensors and Artificial Intelligence Methods and Algorithms for Human–Computer Intelligent Interaction: A Systematic Mapping Study

“…Because automatic emotion recognition has many applications also in HCII, it has attracted the recent hype of AI-empowered HCI research [ 44 ]. Another essential and challenging task related to emotion recognition in HCII is speech emotion recognition [ 45 ], which has become the heart of most HCI applications in the modern world [ 46 ]. For many years, eye-tracking technology has been used for usability testing and implementation of various solutions for controlling the user interface.…”

“…Researchers have developed different methods and algorithms for analyzing the emotional condition of an individual user with the focus on emotion classification by salient acoustic features of speech. Most researchers in speech emotion recognition have applied handcrafted features and machine learning techniques in recognizing speech emotion [ 46 ]. In existing speech emotion recognition research, classical ML classifiers were used, such as the Markov model (MM), Gaussian mixed model (GMM), and SVM [ 118 ].…”

“…In existing speech emotion recognition research, classical ML classifiers were used, such as the Markov model (MM), Gaussian mixed model (GMM), and SVM [ 118 ]. Existing research has demonstrated that DLA effectively extract robust and salient features in the dataset [ 46 ]. After the ANN breakthrough, especially CNN, the neural approach has become the main one for creating intelligent computer vision systems [ 119 ].…”

Analysis on the Penetration of Emotional Education in College Physical Education Based on Emotional Feature Clustering

“…e emotion recognition algorithm based on emotion feature clustering is based on an observation: words or phrases of emotional education in college physical education with the same emotional tendency are more likely to appear in the same product comments [11,12]. For example, in general, the probability of "very good" and "perfect" appearing in the same comment with a favorable emotional tendency is higher than that of "very poor" and "perfect" appearing in a comment together [13].…”

Speech emotion recognition method in educational scene based on machine learning

“…As a nonverbal behavior contained in sound, phonological emotion plays an important role in assisting knowledge teaching and transmitting teaching effect feedback [5], which has also attracted the attention of more and more researchers. Zvarevashe and olugbara studied speech emotion recognition method based on twodimensional convolutional neural network deep learning algorithm [6], and developed a customized twodimensional convolutional neural network, which can extract and classify speech features at the same time. The research shows that the deep learning algorithm can effectively extract the robust salient features in the data set.…”