Emotional interactive movie: adjusting the scenario according to the emotional response of the viewer

In recent years, emotion recognition has received significant attention, presenting a plethora of opportunities for application in diverse fields such as human–computer interaction, psychology, and neuroscience, to name a few. Although unimodal emotion recognition methods offer certain benefits, they have limited ability to encompass the full spectrum of human emotional expression. In contrast, Multimodal Emotion Recognition (MER) delivers a more holistic and detailed insight into an individual's emotional state. However, existing multimodal data collection approaches utilizing contact-based devices hinder the effective deployment of this technology. We address this issue by examining the potential of contactless data collection techniques for MER. In our tertiary review study, we highlight the unaddressed gaps in the existing body of literature on MER. Through our rigorous analysis of MER studies, we identify the modalities, specific cues, open datasets with contactless cues, and unique modality combinations. This further leads us to the formulation of a comparative schema for mapping the MER requirements of a given scenario to a specific modality combination. Subsequently, we discuss the implementation of Contactless Multimodal Emotion Recognition (CMER) systems in diverse use cases with the help of the comparative schema which serves as an evaluation blueprint. Furthermore, this paper also explores ethical and privacy considerations concerning the employment of contactless MER and proposes the key principles for addressing ethical and privacy concerns. The paper further investigates the current challenges and future prospects in the field, offering recommendations for future research and development in CMER. Our study serves as a resource for researchers and practitioners in the field of emotion recognition, as well as those intrigued by the broader outcomes of this rapidly progressing technology.

show abstract

The Generative Fairy Tale of Scary Little Red Riding Hood

Harde,

Jensen,

Krogh

et al. 2024

ACM International Conference on Interactive Media Experiences

View full text Add to dashboard Cite

Emotional interactive movie: adjusting the scenario according to the emotional response of the viewer

Cited by 4 publications

References 20 publications

Methodology for Detection of ERD/ERS EEG Patterns Produced by Cut Events in Film Fragments

Methodology for Detection of ERD/ERS EEG Patterns Produced by Cut Events in Film Fragments

Exploring contactless techniques in multimodal emotion recognition: insights into diverse applications, challenges, solutions, and prospects

The Generative Fairy Tale of Scary Little Red Riding Hood

Contact Info

Product

Resources

About