An ensemble 1D-CNN-LSTM-GRU model with data augmentation for speech emotion recognition

Ahmed, Md. Rayhan; Islam, Salekul; Islam, A.K.M. Muzahidul; Shatabda, Swakkhar

doi:10.1016/j.eswa.2023.119633

Cited by 45 publications

(30 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The process of image augmentation introduces diversity to images, thereby enhancing the overall generalizability and efficacy of ML and DL-based classification models [14] . To augment the number of images we have utilized the Keras ImageDataGenerator class.…”

Section: Experimental Design Materials and Methodsmentioning

confidence: 99%

BDMediLeaves: A leaf images dataset for Bangladeshi medicinal plants identification

Islam,

Ahmed,

Islam

et al. 2023

Data in Brief

View full text Add to dashboard Cite

Section: Experimental Design Materials and Methodsmentioning

confidence: 99%

BDMediLeaves: A leaf images dataset for Bangladeshi medicinal plants identification

Islam,

Ahmed,

Islam

et al. 2023

Data in Brief

View full text Add to dashboard Cite

“…According to the findings presented by [35] acknowledge that the former investigators do not accomplish capturing a voice signal's global, long-term context since they only recover local hidden facets. Due to limited dataset availability and inadequate feature portrayals, they exhibit poor recognition performance.…”

Section: Related Workmentioning

confidence: 99%

XEmoAccent: Embracing Diversity in Cross-Accent Emotion Recognition Using Deep Learning

Ahmad,

Iqbal,

Mohsin Jadoon

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Speech is a powerful means of expressing thoughts, emotions, and perspectives. However, accurately determining the emotions conveyed through speech remains a challenging task. Existing manual methods for analyzing speech to recognize emotions are prone to errors, limiting our understanding and response to individuals' emotional states. To address diverse accents, an automated system capable of realtime emotion prediction from human speech is needed. This paper introduces a speech emotion recognition (SER) system that leverages supervised learning techniques to tackle cross-accent diversity. Distinctively, the system extracts a comprehensive set of nine speech features-Zero Crossing Rate, Mel Spectrum, Pitch, Root Mean Square values, Mel Frequency Cepstral Coefficients, chroma-stft, and three spectral features (Centroid, Contrast, and Roll-off) for refined speech signal processing and recognition. Seven machine learning models are employed, encompassing Random Forest, Logistic Regression, Decision Tree, Support Vector Machines, Gaussian Naive Bayes, K-Nearest Neighbors, ensemble learning, and four individual, hybrid deep learning models including Long short-term memory (LSTM) and 1-Dimensional Convolutional Neural Network (1D-CNN) with stratified cross-validation. Audio samples from diverse English regions are combined to train the models. The performance evaluation results of conventional machine learning and deep learning models indicate that the Random Forest-based feature selection model achieves the highest accuracy of up to 76% among the conventional machine learning models. Simultaneously, the 1D-CNN model with stratified cross-validation reaches up to 99% accuracy. The proposed framework enhances the cross-accent emotion recognition accuracy up to 86.3%, 89.87%, 90.27%, and 84.96% by margins of 14.71%, 10.15%, 9.6%, and 16.52% respectively.

show abstract

“…An ensemble model by combining Convolutional neural network, long short-term memory, and gated recurrent unit for speech emotion recognition has been proposed in [30]. Despite this extensive research works, there is an imperative need to construct novel Convolutional neural network models with a focus on shortening the training time and for improved accuracy.…”

Section: Related Workmentioning

confidence: 99%

A Novel Convolutional Neural Network Model for Automatic Speaker Identification From Speech Signals

Pandian,

Thirunavukarasu,

Kotei

2024

IEEE Access

View full text Add to dashboard Cite

Since voice communication is the most frequent form of human communication, it is necessary to identify each speaker in a discussion. The development of advanced computer processing units, deep learning techniques and publicly available datasets prompt the need for proposing novel automated speaker recognition systems. This study presents a novel Convolutional Neural Network (CNN) called Five Convolutional Blocks-CNN (5C-CNN) for automatic speaker identification from speech signals. The proposed 5C-CNN model comprises five Convolutional blocks and one dense block for identifying the speaker using their English speech audio. The proposed framework was trained with English audio speech signals collected from Ten individuals. The study also applied different audio augmentation techniques like white Gaussian noise, shifting, stretching, stretching with high frequency, altering speed, and changing pitch values to introduce diversity in the dataset. HyperBand tuning technique was used to optimize the training parameters of the proposed approach. The proposed 5C-CNN achieved a classification accuracy of 99.34% on the test data of the proposed person speech dataset. Also, the proposed approach performed well when tested against a benchmark dataset namely THUYG-20 for better generalization with an accuracy of 95.43%. Both experimental investigations clearly confirm the efficiency of the proposed technique in the audio classification task while comparing the existing approaches, and standard machine learning techniques.

show abstract

An ensemble 1D-CNN-LSTM-GRU model with data augmentation for speech emotion recognition

Cited by 45 publications

References 89 publications

BDMediLeaves: A leaf images dataset for Bangladeshi medicinal plants identification

BDMediLeaves: A leaf images dataset for Bangladeshi medicinal plants identification

XEmoAccent: Embracing Diversity in Cross-Accent Emotion Recognition Using Deep Learning

A Novel Convolutional Neural Network Model for Automatic Speaker Identification From Speech Signals

Contact Info

Product

Resources

About