Newman-Watts-Strogatz topology in deep echo state networks for speech emotion recognition

Soltani, Rebh; Benmohamed, Emna; Ltifi, Hela

doi:10.1016/j.engappai.2024.108293

Cited by 2 publications

(1 citation statement)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep-CNN [64] 71.61 2020 CNN-LSTM [65] 53.08 2020 QCNN [66] 77.87 2021 TSP-INCA [60] 87.43 87.43 2021 Wav2Vec2-PT [40] 84.3 2021 MLAnet [69] 84.48 83.55 2022 TIM-Net [62] 92.08 91.93 2023 DS-CTFN [46] 97.7 97.6 2023 NWS graph [67] 76.45 75.79 2024 (HuBERT + DPCNN) + CAF 81.86 82.75 2024…”

Section: Wa (%) Ua (%) Yearmentioning

confidence: 99%

Speech Emotion Recognition Using Dual-Stream Representation and Cross-Attention Fusion

Yu,

Meng,

Fan

et al. 2024

Electronics

View full text Add to dashboard Cite

Speech emotion recognition (SER) aims to recognize human emotions through in-depth analysis of audio signals. However, it remains challenging to encode emotional cues and to fuse the encoded cues effectively. In this study, dual-stream representation is developed, and both full training and fine-tuning of different deep networks are employed for encoding emotion patterns. Specifically, a cross-attention fusion (CAF) module is designed to integrate the dual-stream output for emotion recognition. Using different dual-stream encoders (fully training a text processing network and fine-tuning a pre-trained large language network), the CAF module is compared to other three fusion modules on three databases. The SER performance is quantified with weighted accuracy (WA), unweighted accuracy (UA), and F1-score (F1S). The experimental results suggest that the CAF outperforms the other three modules and leads to promising performance on the databases (EmoDB: WA, 97.20%; UA, 97.21%; F1S, 0.8804; IEMOCAP: WA, 69.65%; UA, 70.88%; F1S, 0.7084; RAVDESS: WA, 81.86%; UA, 82.75.21%; F1S, 0.8284). It is also found that fine-tuning a pre-trained large language network achieves superior representation than fully training a text processing network. In a future study, improved SER performance could be achieved through the development of a multi-stream representation of emotional cues and the incorporation of a multi-branch fusion mechanism for emotion recognition.

show abstract

Section: Wa (%) Ua (%) Yearmentioning

confidence: 99%

Speech Emotion Recognition Using Dual-Stream Representation and Cross-Attention Fusion

Yu,

Meng,

Fan

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

MelTrans: Mel-Spectrogram Relationship-Learning for Speech Emotion Recognition via Transformers

Li,

Liu

et al. 2024

Sensors

View full text Add to dashboard Cite

Speech emotion recognition (SER) is not only a ubiquitous aspect of everyday communication, but also a central focus in the field of human–computer interaction. However, SER faces several challenges, including difficulties in detecting subtle emotional nuances and the complicated task of recognizing speech emotions in noisy environments. To effectively address these challenges, we introduce a Transformer-based model called MelTrans, which is designed to distill critical clues from speech data by learning core features and long-range dependencies. At the heart of our approach is a dual-stream framework. Using the Transformer architecture as its foundation, MelTrans deciphers broad dependencies within speech mel-spectrograms, facilitating a nuanced understanding of emotional cues embedded in speech signals. Comprehensive experimental evaluations on the EmoDB (92.52%) and IEMOCAP (76.54%) datasets demonstrate the effectiveness of MelTrans. These results highlight MelTrans’s ability to capture critical cues and long-range dependencies in speech data, setting a new benchmark within the context of these specific datasets. These results highlight the effectiveness of the proposed model in addressing the complex challenges posed by SER tasks.

show abstract

Newman-Watts-Strogatz topology in deep echo state networks for speech emotion recognition

Cited by 2 publications

References 35 publications

Speech Emotion Recognition Using Dual-Stream Representation and Cross-Attention Fusion

Speech Emotion Recognition Using Dual-Stream Representation and Cross-Attention Fusion

MelTrans: Mel-Spectrogram Relationship-Learning for Speech Emotion Recognition via Transformers

Contact Info

Product

Resources

About