Recognition of Emotion in Speech-related Audio Files with LSTM-Transformer

Andayani, Felicia; Tsun, Mark TeeKit; Chua, Caslon

doi:10.1109/icci54321.2022.9756100

2022 5th International Conference on Computing and Informatics (ICCI) 2022

DOI: 10.1109/icci54321.2022.9756100

|View full text |Cite

Recognition of Emotion in Speech-related Audio Files with LSTM-Transformer

Felicia Andayani

Mark TeeKit Tsun

Caslon Chua

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

2024

Publication Types

Select...

Article3

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

MVIB-DVA: Learning minimum sufficient multi-feature speech emotion embeddings under dual-view aware

Li,

Hou,

Liu

et al. 2024

Expert Systems with Applications

View full text Add to dashboard Cite

MVIB-DVA: Learning minimum sufficient multi-feature speech emotion embeddings under dual-view aware

Li,

Hou,

Liu

et al. 2024

Expert Systems with Applications

View full text Add to dashboard Cite

Mel-MViTv2: Enhanced Speech Emotion Recognition With Mel Spectrogram and Improved Multiscale Vision Transformers

Ong,

Lee,

Lim

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Speech emotion recognition aims to automatically identify and classify emotions from speech signals. It plays a crucial role in various applications such as human-computer interaction, affective computing, and social robotics. Over the years, researchers have proposed different approaches for speech emotion recognition, leveraging various classifiers and features. However, despite the advancements, existing methods in speech emotion recognition still have certain limitations. Some approaches rely on handcrafted features that may not capture the full complexity of emotional information present in speech signals, while others may suffer from a lack of robustness and generalization when applied to different datasets. To address these challenges, this paper proposes a speech emotion recognition method that combines Mel spectrogram with Short-Term Fourier Transform (Mel-STFT) and the Improved Multiscale Vision Transformers (MViTv2). The Mel-STFT spectrograms capture both the frequency and temporal information of speech signals, providing a more comprehensive representation of the emotional content. The MViTv2 classifier introduces multi-scale visual modeling with different stages and pooling attention mechanisms. MViTv2 incorporates relative positional embeddings and a residual pooling connection to effectively model the interactions between tokens in the space-time structure, preserve essential information, and improve the efficiency of the model. Experimental results demonstrate that the proposed method generalizes well on different datasets, achieving an accuracy of 91.51% on the Emo-DB dataset, 81.75% on the RAVDESS dataset, and 64.03% on the IEMOCAP dataset.

show abstract

Memristor-Based Progressive Hierarchical Conformer Architecture for Speech Emotion Recognition

Zhao,

Zhou,

2024

Int. J. Bifurcation Chaos

View full text Add to dashboard Cite

Speech Emotion Recognition (SER) is a challenging task characterized by the diversity and complexity of emotional expression. Due to its powerful feature extraction capabilities, Transformer Network (TN) demonstrates advantages and potential in SER. However, the limited size of available datasets and the difficulty of decoupling emotional features restrain its performance and present challenges in implementing SER on edge devices. To address these issues, we present a Memristor-based Progressive Hierarchical Conformer Architecture (MPCA) and design a conformer submodule that leverages convolution to mitigate TN’s limitations in SER. We propose attention-based feature decoupling, employing hierarchical extraction to decouple speaker characteristics and retain the relevant components, thereby obtaining reliable emotional features. Furthermore, we propose a reconfigurable circuit implementation scheme for MPCA based on operator multiplexing achieving flexible modules that can be dynamically adjusted based on the resources of edge devices, and the stability of the designed circuit is analyzed by simulation experiments with PSPICE. We show that the suggested MPCA demonstrates state-of-the-art performance in SER while significantly reducing system power consumption, offering a solution for SER implementation on edge devices.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Recognition of Emotion in Speech-related Audio Files with LSTM-Transformer

Cited by 3 publications

References 15 publications

MVIB-DVA: Learning minimum sufficient multi-feature speech emotion embeddings under dual-view aware

MVIB-DVA: Learning minimum sufficient multi-feature speech emotion embeddings under dual-view aware

Mel-MViTv2: Enhanced Speech Emotion Recognition With Mel Spectrogram and Improved Multiscale Vision Transformers

Memristor-Based Progressive Hierarchical Conformer Architecture for Speech Emotion Recognition

Contact Info

Product

Resources

About