Speech Emotion Recognition Based on Multiple Acoustic Features and Deep Convolutional Neural Network

Bhangale, Kishor; Kothandaraman, Mohanaprasad

doi:10.3390/electronics12040839

Cited by 40 publications

(20 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2 constitute the time domain feature, which shows the impact of sentiment on the temporal properties of speech. The voice quality features jitter, root mean square (RMS), and shimmer value, which provide variations in voice quality due to emotions in terms of amplitude and time [7]. The details of the features are described in Table 1.…”

Section: Methodsmentioning

confidence: 99%

“…Instead of relying on conventional input devices, this system uses voice commands to communicate naturally between the user and the computer [1][2][3]. There are a wide variety of applications for SSR, including but not limited to interactive robotics, contact centers, onboard vehicle driving systems, interactive game development, online learning, medical-psychological analysis, and tutoring systems [4][5][6][7].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Speech-based Sentiment Recognition System using PDCNN and LSTM Algorithms

Adagale,

Gupta

2024

Preprint

View full text Add to dashboard Cite

Sentiment analysis is very important for the multiple human-computer interaction system. Many deep learning frameworks have been presented for sentiment analysis using speech signals. However, the performance of speech-based sentiment recognition is limited because of poor generalization capability, limited long-term dependency, inferior feature representation, poor balance in speech spectral and temporal properties, and complexity in deep learning algorithm frameworks. This paper presents speech-based sentiment recognition (SSR) using a parallel deep convolution neural network, a long short-term memory (DCNN-LSTM) network, and multiple acoustic features (MAF). The multiple acoustic features consist of spectral, time domain, and voice quality features for improving the feature distinctiveness of the speech signal. Further, the Archimedes Optimization algorithm (AoA) selects the prominent MAFs. The weights of the fitness function of the AoA algorithms are automatically optimized using the Multi-Attribute Criteria Theory (MAUT) algorithm. The outcomes of the proposed algorithm are evaluated on the Berlin Sentiment Database (EMODB), which consists of seven sentiments: happiness, anger, boredom, disgust, fear, neutral, and sadness.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Speech-based Sentiment Recognition System using PDCNN and LSTM Algorithms

Adagale,

Gupta

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The authors of [ 41 ] proposed a methodology for SER that leverages MFCC and a one-dimensional convolutional neural network with the aim of diminishing computational complexity. The approach involves the use of various acoustic properties to present collaborative low-order and high-order features and the development of a lightweight one-dimensional deep convolutional neural network to streamline the deep learning frameworks for SER.…”

Section: Methodsmentioning

confidence: 99%

Enhancing Speech Emotion Recognition Using Dual Feature Extraction Encoders

Pulatov,

Oteniyazov,

Makhmudov

et al. 2023

Sensors

View full text Add to dashboard Cite

Understanding and identifying emotional cues in human speech is a crucial aspect of human–computer communication. The application of computer technology in dissecting and deciphering emotions, along with the extraction of relevant emotional characteristics from speech, forms a significant part of this process. The objective of this study was to architect an innovative framework for speech emotion recognition predicated on spectrograms and semantic feature transcribers, aiming to bolster performance precision by acknowledging the conspicuous inadequacies in extant methodologies and rectifying them. To procure invaluable attributes for speech detection, this investigation leveraged two divergent strategies. Primarily, a wholly convolutional neural network model was engaged to transcribe speech spectrograms. Subsequently, a cutting-edge Mel-frequency cepstral coefficient feature abstraction approach was adopted and integrated with Speech2Vec for semantic feature encoding. These dual forms of attributes underwent individual processing before they were channeled into a long short-term memory network and a comprehensive connected layer for supplementary representation. By doing so, we aimed to bolster the sophistication and efficacy of our speech emotion detection model, thereby enhancing its potential to accurately recognize and interpret emotion from human speech. The proposed mechanism underwent a rigorous evaluation process employing two distinct databases: RAVDESS and EMO-DB. The outcome displayed a predominant performance when juxtaposed with established models, registering an impressive accuracy of 94.8% on the RAVDESS dataset and a commendable 94.0% on the EMO-DB dataset. This superior performance underscores the efficacy of our innovative system in the realm of speech emotion recognition, as it outperforms current frameworks in accuracy metrics.

show abstract

“…The RMS value can be calculated for short-time windows of the speech signal, typically in the range of 20-50 milliseconds. The RMS values for these short-time windows can be used to characterize the changes in loudness or energy over time, which can be indicative of changes in emotional content [43].…”

Section: Root Mean Squarementioning

confidence: 99%

Real-Time Speech Emotion Recognition Using Deep Learning and Data Augmentation

Barhoumi¹,

Ayed

2023

Preprint

View full text Add to dashboard Cite

In human-human interactions, detecting emotions is often easy as it can be perceived through facial expressions, body gestures, or speech. However, in human-machine interactions, detecting human emotion can be a challenge. To improve this interaction, the term 'speech emotion recognition' has emerged, with the goal of recognizing emotions solely through vocal intonation. In this work, we propose a speech emotion recognition system based on deep learning approaches and two efficient data augmentation techniques (noise addition and spectrogram shifting). To evaluate the proposed system, we used three different datasets: TESS, EmoDB, and RAVDESS. We employe several algorithms such as Mel Frequency Cepstral Coefficients (MFCC), Zero Crossing Rate (ZCR), Mel spectrograms, Root Mean Square Value (RMS), and chroma to select the most appropriate vocal features that represent speech emotions. To develop our speech emotion recognition system, we use three different deep learning models, including MultiLayer Perceptron (MLP), Convolutional Neural Network (CNN), and a hybrid model that combines CNN with Bidirectional Long-Short Term Memory (Bi-LSTM). By exploring these different approaches, we were able to identify the most effective model for accurately identifying emotional states from speech signals in real-time situation. Overall, our work demonstrates the effectiveness of the proposed deep learning model, specifically based on CNN+BiLSTM, and the used two data augmentation techniques for the proposed real-time speech emotion recognition.

show abstract

Speech Emotion Recognition Based on Multiple Acoustic Features and Deep Convolutional Neural Network

Cited by 40 publications

References 41 publications

Speech-based Sentiment Recognition System using PDCNN and LSTM Algorithms

Speech-based Sentiment Recognition System using PDCNN and LSTM Algorithms

Enhancing Speech Emotion Recognition Using Dual Feature Extraction Encoders

Real-Time Speech Emotion Recognition Using Deep Learning and Data Augmentation

Contact Info

Product

Resources

About