Deep Learning Approaches for Speech Emotion Recognition

Bhavan, Anjali; Sharma, Mohit; Piplani, Mehak; Chauhan, Pankaj; Hitkul,; Shah, Rajiv Ratn

doi:10.1007/978-981-15-1216-2_10

Cited by 6 publications

(4 citation statements)

References 86 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RAVDESS is a very recent dataset, and has been adopted in research so far mainly in the field of emotion detection from speech. State of the art accuracy on the dataset for the problem reaches over 70% (see [43][44][45][46][47] for an overview). However, to our knowledge there is no work on visual only emotion recognition on the RAVDESS dataset.…”

Section: Datasetmentioning

confidence: 99%

Improving the Accuracy of Automatic Facial Expression Recognition in Speaking Subjects with Deep Learning

et al. 2020

View full text Add to dashboard Cite

When automatic facial expression recognition is applied to video sequences of speaking subjects, the recognition accuracy has been noted to be lower than with video sequences of still subjects. This effect known as the speaking effect arises during spontaneous conversations, and along with the affective expressions the speech articulation process influences facial configurations. In this work we question whether, aside from facial features, other cues relating to the articulation process would increase emotion recognition accuracy when added in input to a deep neural network model. We develop two neural networks that classify facial expressions in speaking subjects from the RAVDESS dataset, a spatio-temporal CNN and a GRU cell RNN. They are first trained on facial features only, and afterwards both on facial features and articulation related cues extracted from a model trained for lip reading, while varying the number of consecutive frames provided in input as well. We show that using DNNs the addition of features related to articulation increases classification accuracy up to 12%, the increase being greater with more consecutive frames provided in input to the model.

show abstract

Section: Datasetmentioning

confidence: 99%

Improving the Accuracy of Automatic Facial Expression Recognition in Speaking Subjects with Deep Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“… Among classification algorithms the most common choices are: naïve Bayes [158,159,160], Decision Tree [161,162,163], Random Forest [164,165,166], Support Vector Machines [167,168,169], and K Nearest Neighbors [170,171,172].  Among regression algorithms the usual choices are: linear regression [173,174,175], Lasso Regression [176,177], Logistic Regression [178,179,180], Multivariate Regression [181,182], and Multiple Regression Algorithm [183,184].…”

Section: Classificationsmentioning

confidence: 99%

Internet and Biometric Web Based Business Management Decision Support

Kaklauskas¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Today, the trend is evolving to the use of certain well-known features as MFCCs with simple models, as in the work of Bhavan et al [38]. They use MFCCs with spectral centroids as input features, introduced into a bagged ensemble of Support Vector Machines, achieving an overall accuracy of 72.91% for RAVDESS.…”

Section: Speech Emotion Recognitionmentioning

confidence: 99%

Multimodal Emotion Recognition on RAVDESS Dataset Using Transfer Learning

Jiménez

Griol

Callejas

et al. 2021

Sensors

View full text Add to dashboard Cite

Emotion Recognition is attracting the attention of the research community due to the multiple areas where it can be applied, such as in healthcare or in road safety systems. In this paper, we propose a multimodal emotion recognition system that relies on speech and facial information. For the speech-based modality, we evaluated several transfer-learning techniques, more specifically, embedding extraction and Fine-Tuning. The best accuracy results were achieved when we fine-tuned the CNN-14 of the PANNs framework, confirming that the training was more robust when it did not start from scratch and the tasks were similar. Regarding the facial emotion recognizers, we propose a framework that consists of a pre-trained Spatial Transformer Network on saliency maps and facial images followed by a bi-LSTM with an attention mechanism. The error analysis reported that the frame-based systems could present some problems when they were used directly to solve a video-based task despite the domain adaptation, which opens a new line of research to discover new ways to correct this mismatch and take advantage of the embedded knowledge of these pre-trained models. Finally, from the combination of these two modalities with a late fusion strategy, we achieved 80.08% accuracy on the RAVDESS dataset on a subject-wise 5-CV evaluation, classifying eight emotions. The results revealed that these modalities carry relevant information to detect users’ emotional state and their combination enables improvement of system performance.

show abstract

Deep Learning Approaches for Speech Emotion Recognition

Cited by 6 publications

References 86 publications

Improving the Accuracy of Automatic Facial Expression Recognition in Speaking Subjects with Deep Learning

Improving the Accuracy of Automatic Facial Expression Recognition in Speaking Subjects with Deep Learning

Internet and Biometric Web Based Business Management Decision Support

Multimodal Emotion Recognition on RAVDESS Dataset Using Transfer Learning

Contact Info

Product

Resources

About