A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS Dataset

Jiménez, Cristina Luna; Kleinlein, Ricardo; Griol, David; Callejas, Zoraida; Montero, Juan Manuel; Fernández-Martínez, Fernando

doi:10.3390/app12010327

Cited by 42 publications

(12 citation statements)

References 69 publications

(78 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a new type of therapy, music therapy takes the function of music as the basis and intervenes the performers through various forms of music, so as to relieve the performers' artistic psychological intervention. This paper studies and analyzes the psychological intervention of music therapy in vocal music performance students' artistic psychological intervention [ 4 – 6 ].…”

Section: Introductionmentioning

confidence: 99%

Design of Interactive Vocal Guidance and Artistic Psychological Intervention System Based on Emotion Recognition

Yuan

2022

Occupational Therapy International

View full text Add to dashboard Cite

The research on artistic psychological intervention to judge emotional fluctuations by extracting emotional features from interactive vocal signals has become a research topic with great potential for development. Based on the interactive vocal music instruction theory of emotion recognition, this paper studies the design of artistic psychological intervention system. This paper uses the vocal music emotion recognition algorithm to first train the interactive recognition network, in which the input is a row vector composed of different vocal music characteristics, and finally recognizes the vocal music of different emotional categories, which solves the problem of low data coupling in the artistic psychological intervention system. Among them, the vocal music emotion recognition experiment based on the interactive recognition network is mainly carried out from six aspects: the number of iterative training, the vocal music instruction rate, the number of emotion recognition signal nodes in the artistic psychological intervention layer, the number of sample sets, different feature combinations, and the number of emotion types. The input data of the system is a training class learning video, and actions and expressions need to be recognized before scoring. In the simulation process, before the completion of the sample indicators is unbalanced, the R language statistical analysis tool is used to balance the existing unbalanced data based on the artificial data synthesis method, and 279 uniformly classified samples are obtained. The 279 ∗ 7 dataset was used for statistical identification of the participants. The experimental results show that under the guidance of four different interactive vocal music, the vocal emotion recognition rate is between 65.85%-91.00%, which promotes the intervention of music therapy on artistic psychological intervention.

show abstract

Section: Introductionmentioning

confidence: 99%

Design of Interactive Vocal Guidance and Artistic Psychological Intervention System Based on Emotion Recognition

Yuan

2022

Occupational Therapy International

View full text Add to dashboard Cite

show abstract

“…Among its advantages is having a proportional number of files in each emotion, which avoids the problems caused by training algorithms with unbalanced data. In addition, RAVDEESS is a reference dataset in the research community that has been used in several works [23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39].…”

Section: Databasementioning

confidence: 99%

Facial and speech Emotional Recognition based on Improved Deep Model

Shahrestani,

Motamed,

Yamaghani

2024

Preprint

View full text Add to dashboard Cite

In recent years, emotion recognition as a new method for natural human-computer interaction has attracted the attention of many researchers. Because the automatic recognition of emotion from speech or facial expressions alone has uncertainties; it is expected that emotion recognition based on the fusion of audio-visual information can be done with better accuracy. The purpose of this article is to present an efficient method for emotion recognition from emotional speech signals and images of emotional facial expressions, based on the SOAR cognitive science model. In this article, in order to implement the subsections of the SOAR model, stochastic deep learning has been used. In the proposed model, the RAVDESS database is used for testing and evaluation, and we have achieved 89.44% accuracy in the evaluation.

show abstract

“…Luna-Jimenez and Kleinlein [36] proposed an automatic emotion-recognizer system that had a speech emotion recognizer (SER) and a facial emotion recognizer (FER). Eight emotions were classified, and they achieved 86.70% accuracy on the RAVDESS dataset using a subject-wise 5-CV evaluation.…”

Section: Review Of Related Literaturementioning

confidence: 99%

Perspective on Dark-Skinned Emotion Recognition Using Deep-Learned and Handcrafted Feature Techniques

Irhebhude¹,

Kolawole²,

Amos³

2023

Emotion Recognition - Recent Advances, New Perspectives and Applications

View full text Add to dashboard Cite

Image recognition has been widely used in various fields of applications such as human—computer interaction, where it can enhance fluency, accuracy, and naturalness in interaction. The need to automate the decision on human expression is high. This paper presents a technique for emotion recognition and classification based on a combination of deep-learned and handcrafted features. Residual Network (ResNet) and Rotation Invariant Local Binary Pattern (RILBP) features were combined and used as features for classification. The aim is to classify, identify, and make judgment on facial images from dark-skinned facial images. Facial Expression Recognition 2013 (FER2013) and self-captured dark-skinned datasets were used for the experiment and validated. The result showed 93.4% accuracy on FER dataset and 95.5% on self-captured dataset, which proved the efficiency of the proposed model.

show abstract

A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS Dataset

Cited by 42 publications

References 69 publications

Design of Interactive Vocal Guidance and Artistic Psychological Intervention System Based on Emotion Recognition

Design of Interactive Vocal Guidance and Artistic Psychological Intervention System Based on Emotion Recognition

Facial and speech Emotional Recognition based on Improved Deep Model

Perspective on Dark-Skinned Emotion Recognition Using Deep-Learned and Handcrafted Feature Techniques

Contact Info

Product

Resources

About