Spectators' Synchronization Detection based on Manifold Representation of Physiological Signals

Proceedings of the 24th ACM International Conference on Multimedia

Κωστούλας

Lombardo

et al. 2016

Self Cite

Detection of emotional and aesthetic highlights is a challenge for the affective understanding of movies. Our assumption is that synchronized spectators' physiological and behavioral reactions occur during these highlights. We propose to employ the periodicity score to capture synchronization among groups of spectators' signals. To uncover the periodicity score's capabilities, we compare it with baseline synchronization measures, such as the nonlinear interdependence and the windowed mutual information. The results show that the periodicity score and the pairwise synchronization measures are able to capture different properties of spectators' synchronization, and they indicate the presence of some types of emotional and aesthetic highlights in a movie based on spectators' electro-dermal and acceleration signals.

Section: Discussionsupporting

confidence: 89%

“…We verify if an increase/a drop of the synchronization of the spectators' physiological and behavioral reactions might appear during a particular type of highlights [16], [11]. Figure 3(a) shows the mean values of the PS which are computed over the spectators' physiological signals for all types of highlights.…”

Section: Resultsmentioning

confidence: 91%

Synchronization among Groups of Spectators for Highlight Detection in Movies

Proceedings of the 24th ACM International Conference on Multimedia

Κωστούλας

Lombardo

et al. 2016

Self Cite

“…To take into account that induced emotions are subjective, we included audience reaction based features, namely statistical features of physiological and behavioral reactions. Also, we assume that each person within a movie audience can display similar behaviors and have similar physiological responses when they are watching a movie together because [18], [50], [55], [56], [57]:…”

Section: Movie Audience Reaction Based Featuresmentioning

confidence: 99%

Recognizing Induced Emotions of Movie Audiences from Multimodal Information

IEEE Trans. Affective Comput.

Tian

Lai

et al. 2021

Self Cite

Recognizing emotional reactions of movie audiences to affective movie content is a challenging task in affective computing. Previous research on induced emotion recognition has mainly focused on using audiovisual movie content. Nevertheless, the relationship between the perceptions of the affective movie content (perceived emotions) and the emotions evoked in the audiences (induced emotions) is unexplored. In this work, we studied the relationship between perceived and induced emotions of movie audiences. Moreover, we investigated multimodal modelling approaches to predict movie induced emotions from movie content based features, as well as physiological and behavioral reactions of movie audiences. To carry out analysis of induced and perceived emotions, we first extended an existing database for movie affect analysis by annotating perceived emotions in a crowd-sourced manner. We find that perceived and induced emotions are not always consistent with each other. In addition, we show that perceived emotions, movie dialogues, and aesthetic highlights are discriminative for movie induced emotion recognition besides spectators' physiological and behavioral reactions. Also, our experiments revealed that induced emotion recognition could benefit from including temporal information and performing multimodal fusion. Moreover, our work deeply investigated the gap between affective content analysis and induced emotion recognition by gaining insight into the relationships between aesthetic highlights, induced emotions, and perceived emotions.

“…Most recent work on aesthetic highlight detection in movies defined and estimated a reaction profile of spectators for identification and interpretation of aesthetic scenes [49], or estimated physiological and behavioral changes of spectators exposed to aesthetic content using the dynamic time warping [48,50]. Furthermore, the manifold representation of multiple spectators' physiological signals was proposed to measure a level of synchronization among them [59], and the periodicity score was applied to physiological and behavioral signals to establish synchronization among groups of spectators [60]. Moreover, synchronization measurement has become an important tool for affective multimedia content analysis.…”

Section: Related Workmentioning

confidence: 99%

Aesthetic Highlight Detection in Movies Based on Synchronization of Spectators’ Reactions

ACM Trans. Multimedia Comput. Commun. Appl.

Κωστούλας

Lombardo

et al. 2018

Self Cite

Detection of aesthetic highlights is a challenge for understanding the affective processes taking place during movie watching. In this paper we study spectators' responses to movie aesthetic stimuli in a social context. Moreover, we look for uncovering the emotional component of aesthetic highlights in movies. Our assumption is that synchronized spectators' physiological and behavioral reactions occur during these highlights because: (i) aesthetic choices of filmmakers are made to elicit specific emotional reactions (e.g. special effects, empathy and compassion toward a character, etc.) and (ii) watching a movie together causes spectators' affective reactions to be synchronized through emotional contagion. We compare different approaches to estimation of synchronization among multiple spectators' signals, such as pairwise, group and overall synchronization measures to detect aesthetic highlights in movies. The results show that the unsupervised architecture relying on synchronization measures is able to capture different properties of spectators' synchronization and detect aesthetic highlights based on both spectators' electrodermal and acceleration signals. We discover that pairwise synchronization measures perform the most accurately independently of the category of the highlights and movie genres. Moreover, we observe that electrodermal signals have more discriminative power than acceleration signals for highlight detection.