A Novel Multi-Window Spectrogram Augmentation Approach for Speech Emotion Recognition Using Deep Learning

Yusuf, Shehu Mohammed; Adedokun, Emmanuel Adewale; Mu’azu, Muhammed Bashir; Umoh, Ime J.; Ibrahim, Ahmed

doi:10.1109/icmeas52683.2021.9692411

Cited by 3 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The spectrogram makes it possible to migrate high-performance CNN models to acoustic spectrogram-based speech emotion recognition because spectrograms can convert 1D sequences into 2D images [10]. For example, Shehu et al [13] proposed an enhancement approach to remove the limitations of overfitting on small training datasets, which employed a multi-windows enhancement strategy to increase the representative number of spectrograms. Satt et.…”

Section: A Acoustic Features Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Video-Based Cross-Modal Auxiliary Network for Multimodal Sentiment Analysis

Chen

Zhou

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Multimodal sentiment analysis has a wide range of applications due to its information complementarity in multimodal interactions. Previous works focus more on investigating efficient joint representations, but they rarely consider the insufficient unimodal features extraction and data redundancy of multimodal fusion. In this paper, a Video-based Cross-modal Auxiliary Network (VCAN) is proposed, which is comprised of an audio features map module and a cross-modal selection module. The first module is designed to substantially increase feature diversity in audio feature extraction, aiming to improve classification accuracy by providing more comprehensive acoustic representations. To empower the model to handle redundant visual features, the second module is addressed to efficiently filter the redundant visual frames during integrating audiovisual data. Moreover, a classifier group consisting of several image classification networks is introduced to predict sentiment polarities and emotion categories. Extensive experimental results on RAVDESS, CMU-MOSI, and CMU-MOSEI benchmarks indicate that VCAN is significantly superior to the state-of-the-art methods for improving the classification accuracy of multimodal sentiment analysis.

show abstract

Section: A Acoustic Features Extractionmentioning

confidence: 99%

“…Additionally, the shortage of available speech samples also affects the accuracy of SER [13]. Some audio signals with strong noise are difficult to directly analyze the emotional components.…”

Section: A Acoustic Features Extractionmentioning

confidence: 99%