A Review Speech Emotion Recognition

Peerzade, G.N.; Deshmukh, Ratnadeep R.; Waghmare, Surendra

doi:10.26438/ijcse/v6i3.400402

Cited by 12 publications

(7 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MFCCs are widely successful in the recognition of various types of audio signals as well as in various human speech processing tasks and are standard in such studies. Some examples include the use of MFCC-based features for language identification systems [ 50 ], speech emotion recognition [ 53 ], and speaker identification [ 65 ]. In [ 50 ], a comprehensive review on the use of MFCC-based features for language identification is presented.…”

Section: Theoretical Backgroundmentioning

confidence: 99%

Robust Detection of COVID-19 in Cough Sounds

Mouawad

Dubnov²,

Dubnov

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

, otherwise known as the coronavirus, has precipitated the world into a pandemic that has infected, as of the time of writing, more than 10 million persons worldwide and caused the death of more than 500,000 persons. Early symptoms of the virus include trouble breathing, fever and fatigue and over 60% of people experience a dry cough. Due to the devastating impact of COVID-19 and the tragic loss of lives, it is of the utmost urgency to develop methods for the early detection of the disease that may help limit its spread as well as aid in the development of targeted solutions. Coughs and other vocal sounds contain pulmonary health information that can be used for diagnostic purposes, and recent studies in chaotic dynamics have shown that nonlinear phenomena exist in vocal signals. The present work investigates the use of symbolic recurrence quantification measures with MFCC features for the automatic detection of COVID-19 in cough sounds of healthy and sick individuals. Our performance evaluation reveals that our symbolic dynamics measures capture the complex dynamics in the vocal sounds and are highly effective at discriminating sick and healthy coughs. We apply our method to sustained vowel 'ah' recordings, and show that our model is robust for the detection of the disease in sustained vowel utterances as well. Furthermore, we introduce a robust novel method of informative undersampling using information rate to deal with the imbalance in our dataset, due to the unavailability of an equal number of sick and healthy recordings. The proposed model achieves a mean classification performance of 97% and 99%, and a mean F 1 -score of 91% and 89% after optimization, for coughs and sustained vowels, respectively.

show abstract

Section: Theoretical Backgroundmentioning

confidence: 99%

Robust Detection of COVID-19 in Cough Sounds

Mouawad

Dubnov²,

Dubnov

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

show abstract

“…We investigate how effective the single modality or combinations of modalities is/are in the multimodal downstream tasks. Take MOSEI Emotion Reconition as example (Table 8), we find speech to be the most effective single modality, which makes sense considering the emotional quality of human speech Peerzade et al (2018). Leveraging dual modalities improves upon using the single-modality, and language-speech is the best-performing of the dual combinations.…”

Section: Single Modalitymentioning

confidence: 92%

i-Code: An Integrative and Composable Multimodal Learning Framework

Yang¹,

Fang²,

C³

et al. 2022

Preprint

View full text Add to dashboard Cite

Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview. Most current pretraining methods, however, are limited to one or two modalities. We present i-Code, a self-supervised pretraining framework where users may flexibly combine the modalities of vision, speech, and language into unified and general-purpose vector representations. In this framework, data from each modality are first given to pretrained single-modality encoders. The encoder outputs are then integrated with a multimodal fusion network, which uses novel attention mechanisms and other architectural innovations to effectively combine information from the different modalities. The entire system is pretrained end-to-end with new objectives including masked modality unit modeling and cross-modality contrastive learning. Unlike previous research using only video for pretraining, the i-Code framework can dynamically process single, dual, and triple-modality data during training and inference, flexibly projecting different combinations of modalities into a single representation space. Experimental results demonstrate how i-Code can outperform state-of-the-art techniques on five video understanding tasks and the GLUE NLP benchmark, improving by as much as 11% and demonstrating the power of integrative multimodal pretraining.

show abstract

“…In this section, related work on ESR is briefly overviewed, presenting how the field has evolved over the years. A more in-depth discussion about different approaches to address the ESR task have been developed and can be found in [ 13 , 14 , 15 , 16 ].…”

Section: Related Workmentioning

confidence: 99%

Robust Multi-Scenario Speech-Based Emotion Recognition System

Zhu-Zhou

Gil-Pita

García-Gómez

et al. 2022

Sensors

View full text Add to dashboard Cite

Every human being experiences emotions daily, e.g., joy, sadness, fear, anger. These might be revealed through speech—words are often accompanied by our emotional states when we talk. Different acoustic emotional databases are freely available for solving the Emotional Speech Recognition (ESR) task. Unfortunately, many of them were generated under non-real-world conditions, i.e., actors played emotions, and recorded emotions were under fictitious circumstances where noise is non-existent. Another weakness in the design of emotion recognition systems is the scarcity of enough patterns in the available databases, causing generalization problems and leading to overfitting. This paper examines how different recording environmental elements impact system performance using a simple logistic regression algorithm. Specifically, we conducted experiments simulating different scenarios, using different levels of Gaussian white noise, real-world noise, and reverberation. The results from this research show a performance deterioration in all scenarios, increasing the error probability from 25.57% to 79.13% in the worst case. Additionally, a virtual enlargement method and a robust multi-scenario speech-based emotion recognition system are proposed. Our system’s average error probability of 34.57% is comparable to the best-case scenario with 31.55%. The findings support the prediction that simulated emotional speech databases do not offer sufficient closeness to real scenarios.

show abstract

A Review Speech Emotion Recognition

Cited by 12 publications

References 2 publications

Robust Detection of COVID-19 in Cough Sounds

Robust Detection of COVID-19 in Cough Sounds

i-Code: An Integrative and Composable Multimodal Learning Framework

Robust Multi-Scenario Speech-Based Emotion Recognition System

Contact Info

Product

Resources

About