Emotion modulates eye movement patterns and subsequent memory for the gist and details of movie scenes

Abstract

A basic question in vision research regards where people look in complex scenes and how this influences their performance in various tasks. Previous studies with static images have demonstrated a close link between where people look and what they remember. Here, we examined the pattern of eye movements when participants watched neutral and emotional clips from Hollywood-style movies. Participants answered multiple-choice memory questions concerning visual and auditory scene details immediately upon viewing 1-min-long neutral or emotional movie clips. Fixations were more narrowly focused for emotional clips, and immediate memory for object details was worse compared to matched neutral scenes, implying preferential attention to emotional events. Although we found the expected correlation between where people looked and what they remembered for neutral clips, this relationship broke down for emotional clips. When participants were subsequently presented with key frames (static images) extracted from the movie clips such that presentation duration of the target objects (TOs) corresponding to the multiple-choice questions was matched and the earlier questions were repeated, more fixations were observed on the TOs, and memory performance also improved significantly, confirming that emotion modulates the relationship between gaze position and memory performance. Finally, in a long-term memory test, old/new recognition performance was significantly better for emotional scenes as compared to neutral scenes. Overall, these results are consistent with the hypothesis that emotional content draws eye fixations and strengthens memory for the scene gist while weakening encoding of peripheral scene details.

As film goes byte: The change from analog to digital film perception.

“…If viewers are able to attend to minute details in the background, they might be less attached to plot and character development. In line with these assumptions, Subramanian, Shankar, Sebe, and Melcher (2014) found empirical evidence that emotions modulate eye movement patterns and subsequent memory for the gist and details of movie scenes. However, more data would be necessary to test this explanation.…”

Watch to Edit: Video Retargeting using Gaze

“…While human attention is influenced by bottom‐up cues, it is also impacted by top‐down cues relating to scene semantics such as faces, spoken dialogue, scene actions and emotions which are integral to the storyline [SSSM14, GSY*15]. Leake et al .…”

“…However, the movie script and the high‐level cues used in these works may not always be available. A number of other works have utilized eye tracking data, which is indicative of the semantic and emotional scene aspects [KSK*10, SSSM14], to infer salient scene content.…”

“…Other advanced methods like content‐aware warping, seam carving then followed [VTP*10]. However, most of these methods rely on bottom‐up saliency derived from computational methods which do not consider high‐level scene semantics such as emotions, which humans are sensitive to [KSK*10, SSSM14]. Recently, Jain et al .…”

“…Other advanced methods like content-aware warping, seam carving then followed [VTP * 10]. However, most of these methods rely on bottomup saliency derived from computational methods which do not consider high-level scene semantics such as emotions, which humans are sensitive to [KSK * 10, SSSM14]. Recently, Jain et al [JSSH15] proposed Gaze Driven Re-editing (GDR), which preserves human preferences in scene content without distortion via user gaze cues and re-edits the original video introducing novel cut, pan and zoom operations.…”