Arabic Automatic Speech Recognition Based on Emotion Detection

Abstract

This work presents a novel emotion recognition via automatic speech recognition (ASR) using a deep feed-forward neural network (DFFNN) for Arabic speech. We present results for the recognition of the three emotions happy, angry, and surprised. The Arabic natural audio dataset (ANAD) is used. Twenty-five low-level descriptors (LLDs) are extracted from the audio signals. Different combination of extracted features is examined. Also, the effect of using the principal component analysis (PCA) technique for dimensionality reduction is examined. For the classification stage, DFFNN is used. Also, the problem of imbalances samples in the dataset is managed by using the borderline-synthetic minority over-sampling technique (B-SMOTE). It is shown from the results thatthe best accuracy is obtained when applying PCA on the extracted features is 98.56 %. Also, the accuracy is 98.33 % when using the combination of all the extracted features. This result is not too much different from the accuracy of using PCA. It is followed by the accuracy of using MFCC and LSF which is 97.79 %. It is noticed that the accuracy is 95.63 % when using LSF features which shows that they are dominant features. The obtained results showed an improvement compared to previous studies.

Speech Emotion Recognition in Multimodal Environments with Transformer: Arabic and English Audio Datasets

“…The researchers used the Arabic Natural Audio Dataset (ANAD) , which was previously employed in [31] Novel emotion recognition for Arabic speech using deep feedforward neural network (DFFNN) achieves 98.56% accuracy with PCA and 98.33% with combined features from ANAD dataset. In [39] evaluate three speaker traits-gender, emotion, and dialect-from Arabic speech, employing multitask learning (MTL).…”

“…For instance, in [19], [29], and [30], researchers utilized speech, text, and mocap data, including sub-modes such as facial expressions, hand gestures, and head rotations, to accurately identify emotions. Furthermore, [31] introduced a groundbreaking transformer-based model named multimodal transformers for audio-visual emotion recognition, overcoming the limitations of RNN and LSTM in capturing long-term dependencies. Three transformer branches are included in this model: audio-video cross-attention, video selfattention, and audio self-attention.…”