Hybrid System for Emotion Recognition Based on Facial Expressions and Body Gesture Recognition

Atanassov, Atanas; Pilev, Dimitar; Tomova, Fani N.; Kuzmanova, Vanya D.

doi:10.1109/icai52893.2021.9639829

Cited by 10 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model can be extended to classify primary and secondary emotions in real-time video data and images. Atanassov considers a hybrid multimodal model for improving human emotion recognition based on facial expression and body gesture recognition [89]. The study extends previous investigations on using pre-trained models of deep-learning neural networks (DNNs) for facial emotion recognition (FER) by adding the emotions extracted from body language.…”

Section: Emotion Recognition Modelsmentioning

confidence: 65%

A Comprehensive Study of Emotional Responses in AI-Enhanced Interactive Installation Art

Chen,

Ibrahim

2023

Sustainability

View full text Add to dashboard Cite

This study presents a comprehensive literature review on the convergence of affective computing, interactive installation art, multi-dimensional sensory stimulation, and artificial intelligence (AI) in measuring emotional responses, demonstrating the potential of artificial intelligence in emotion recognition as a tool for sustainable development. It addresses the problem of understanding emotional response and measurement in the context of interactive installation art under artificial intelligence (AI), emphasizing sustainability as a key factor. The study aims to fill the existing research gaps by examining three key aspects: sensory stimulation, multi-dimensional interactions, and engagement, which have been identified as significant contributors to profound emotional responses in interactive installation art. The proposed approach involves conducting a process analysis of emotional responses to interactive installation art, aiming to develop a conceptual framework that explores the variables influencing emotional responses. This study formulates hypotheses that make specific predictions about the relationships between sensory stimulation, multi-dimensional interactions, engagement, and emotional responses. By employing the ASSURE model combined with experimental design, the research methodology ensures a systematic and comprehensive study implementation. The implications of this project lie in advancing the understanding of emotional experiences in interactive installation art under AI, providing insights into the underlying mechanisms that drive these experiences, and their influence on individual well-being from a sustainable perspective. The contributions of this research include bridging the identified research gaps, refining theoretical frameworks, and guiding the design of more impactful and emotionally resonant interactive artworks with sustainability in mind. This research seeks not only to fill the existing gaps in understanding emotional experiences in interactive installation art, but also to guide the development of immersive and emotionally engaging installations, ultimately advancing the broader field of human–computer interaction, promoting individual well-being, and contribute to sustainable development.

show abstract

Section: Emotion Recognition Modelsmentioning

confidence: 65%

A Comprehensive Study of Emotional Responses in AI-Enhanced Interactive Installation Art

Chen,

Ibrahim

2023

Sustainability

View full text Add to dashboard Cite

show abstract

“…In our paper presented in ICAI21 conference, we analyzed most existing DNN models for body's gestures recognition, again for finding suitable pre-trained BER DNN model [9]. Most of analyzed models were based on ResNet18 and ResNet50 architectures.…”

Section: Choosing a Dnn For Body Emotion Recognitionmentioning

confidence: 99%

Multimodal System for Facial Emotion Recognition Based on Deep Learning

Atanassov,

Pilev,

Tomova

2024

JCTM

View full text Add to dashboard Cite

Emotions are one of the main ways of communication between people and of expressing attitudes towards objects, products, services, etc. They are divided to verbal and non-verbal classes. Human speech and intonation belong to the first class, and to the second (non-verbal) facial and body emotions, known as body language. The subject of this report is the development of multimodal deep learning system intended to recognize facial and body emotions and their relationship with the scene (weather) in which they occur. It is based on three deep learning neural networks (DNN) each one for recognition of facial emotion, body emotion and weather. Combining their results, we improve significantly the final facial emotion recognition (FER) results.

show abstract

“…Driver emotion recognition is usually achieved by analyzing a driver's emotional representation. Representations of human emotions can include facial expressions, language, body posture, and physiological changes [30,31]. In this research on driver emotion recognition, a variety of different measurement methods are used [32].…”

Section: Related Workmentioning

confidence: 99%

Drivers’ Comprehensive Emotion Recognition Based on HAM

Zhou,

Cheng,

Wen

et al. 2023

Sensors

View full text Add to dashboard Cite

Negative emotions of drivers may lead to some dangerous driving behaviors, which in turn lead to serious traffic accidents. However, most of the current studies on driver emotions use a single modality, such as EEG, eye trackers, and driving data. In complex situations, a single modality may not be able to fully consider a driver’s complete emotional characteristics and provides poor robustness. In recent years, some studies have used multimodal thinking to monitor single emotions such as driver fatigue and anger, but in actual driving environments, negative emotions such as sadness, anger, fear, and fatigue all have a significant impact on driving safety. However, there are very few research cases using multimodal data to accurately predict drivers’ comprehensive emotions. Therefore, based on the multi-modal idea, this paper aims to improve drivers’ comprehensive emotion recognition. By combining the three modalities of a driver’s voice, facial image, and video sequence, the six classification tasks of drivers’ emotions are performed as follows: sadness, anger, fear, fatigue, happiness, and emotional neutrality. In order to accurately identify drivers’ negative emotions to improve driving safety, this paper proposes a multi-modal fusion framework based on the CNN + Bi-LSTM + HAM to identify driver emotions. The framework fuses feature vectors of driver audio, facial expressions, and video sequences for comprehensive driver emotion recognition. Experiments have proved the effectiveness of the multi-modal data proposed in this paper for driver emotion recognition, and its recognition accuracy has reached 85.52%. At the same time, the validity of this method is verified by comparing experiments and evaluation indicators such as accuracy and F1 score.

show abstract

Hybrid System for Emotion Recognition Based on Facial Expressions and Body Gesture Recognition

Cited by 10 publications

References 6 publications

A Comprehensive Study of Emotional Responses in AI-Enhanced Interactive Installation Art

A Comprehensive Study of Emotional Responses in AI-Enhanced Interactive Installation Art

Multimodal System for Facial Emotion Recognition Based on Deep Learning

Drivers’ Comprehensive Emotion Recognition Based on HAM

Contact Info

Product

Resources

About