Perceptual Borderline for Balancing Multi-Class Spontaneous Emotional Data

Letaifa, Leila Ben; Torres, M. Inés

doi:10.1109/access.2021.3071485

Cited by 10 publications

(12 citation statements)

References 51 publications

(50 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance of DNN models depends on the volume and quality of the datasets. Therefore, the implementation of heuristic sampling methods compensates for the imbalance in the distribution of affective classes [ 47 , 48 ]. Table 4 shows the results of the DNN models proposed in Section 4.7 for affective detection from the physiological dataset balanced with the K-SMOTE and TL techniques.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Tourist Experiences Recommender System Based on Emotion Recognition with Wearable Data

Santamaría-Granados

Mendoza-Moreno

Chantre-Astaiza

et al. 2021

Sensors

View full text Add to dashboard Cite

The collection of physiological data from people has been facilitated due to the mass use of cheap wearable devices. Although the accuracy is low compared to specialized healthcare devices, these can be widely applied in other contexts. This study proposes the architecture for a tourist experiences recommender system (TERS) based on the user’s emotional states who wear these devices. The issue lies in detecting emotion from Heart Rate (HR) measurements obtained from these wearables. Unlike most state-of-the-art studies, which have elicited emotions in controlled experiments and with high-accuracy sensors, this research’s challenge consisted of emotion recognition (ER) in the daily life context of users based on the gathering of HR data. Furthermore, an objective was to generate the tourist recommendation considering the emotional state of the device wearer. The method used comprises three main phases: The first was the collection of HR measurements and labeling emotions through mobile applications. The second was emotional detection using deep learning algorithms. The final phase was the design and validation of the TERS-ER. In this way, a dataset of HR measurements labeled with emotions was obtained as results. Among the different algorithms tested for ER, the hybrid model of Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks had promising results. Moreover, concerning TERS, Collaborative Filtering (CF) using CNN showed better performance.

show abstract

Section: Resultsmentioning

confidence: 99%

“…For this purpose, some studies have used heuristic sampling methods and oversampling techniques for the Multi-class Imbalanced Classification (MIC) using neural networks [ 47 , 48 , 49 ]. These sampling techniques are based on the nearest neighbor rule of the feature space of each class [ 49 , 50 ].…”

Section: Methodsmentioning

confidence: 99%

Tourist Experiences Recommender System Based on Emotion Recognition with Wearable Data

Santamaría-Granados

Mendoza-Moreno

Chantre-Astaiza

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Furthermore, emotions expressed during acting or during a real-life scenario show significant differences [20]. In fact, only a small set of complex and compound emotions [21] can be found in real scenarios [2,15,22], and this subset is strongly dependent on the situation. Therefore, a set of categories including the emotions that arise in each specific task has to be defined.…”

Section: Introductionmentioning

confidence: 99%

“…In fact, the emotional label assigned by a speaker to their own utterances might differ from those assigned by a listener, with the former being, of course, more accurate [26]. In this work, we draw from some works dealing with the annotation of a virtual agent [22,27] that provide insights into the problems associated with this kind of annotation. The intrinsic subjectivity of this task makes obtaining a ground truth for emotional states associated with an audio signal using either the categorical or the dimensional model difficult.…”

Section: Introductionmentioning

confidence: 99%

Automatic Identification of Emotional Information in Spanish TV Debates and Human–Machine Interactions

2022

Self Cite

View full text Add to dashboard Cite

Automatic emotion detection is a very attractive field of research that can help build more natural human–machine interaction systems. However, several issues arise when real scenarios are considered, such as the tendency toward neutrality, which makes it difficult to obtain balanced datasets, or the lack of standards for the annotation of emotional categories. Moreover, the intrinsic subjectivity of emotional information increases the difficulty of obtaining valuable data to train machine learning-based algorithms. In this work, two different real scenarios were tackled: human–human interactions in TV debates and human–machine interactions with a virtual agent. For comparison purposes, an analysis of the emotional information was conducted in both. Thus, a profiling of the speakers associated with each task was carried out. Furthermore, different classification experiments show that deep learning approaches can be useful for detecting speakers’ emotional information, mainly for arousal, valence, and dominance levels, reaching a 0.7F1-score.

show abstract

“…) and the dimensional one (valence, arousal and dominance)[4] -Emotion Recognition from participant's faces in the categorical space. -The multimodal fusion module is in charge of taking emotional status predictions from the other two independent emotional modules (face and speech), in order to fuse them into a singular emotional state score.…”

mentioning

confidence: 99%

The EMPATHIC Virtual Coach: a demo

Olaso

Vázquez

Letaifa

et al. 2021

Proceedings of the 2021 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Perceptual Borderline for Balancing Multi-Class Spontaneous Emotional Data

Abstract: This work has been supported by the European H2020 program trough the EMPATHIC project and the MENHIR MSCA action under grants 769872 and 823907 respectively.

Cited by 10 publications

References 51 publications

Tourist Experiences Recommender System Based on Emotion Recognition with Wearable Data

Tourist Experiences Recommender System Based on Emotion Recognition with Wearable Data

Automatic Identification of Emotional Information in Spanish TV Debates and Human–Machine Interactions

The EMPATHIC Virtual Coach: a demo

Contact Info

Product

Resources

About