Event Segmentation in the Audio Description of Films

Abstract

To make the content of films available to a visually impaired audience, a sighted translator can provide audio description (AD), a verbal description of visual events. To achieve this goal, the audio describer needs to select what to describe, when to describe it, and how to describe it, as well as to express the information aurally. The efficacy of this communication is critically dependent upon basic cognitive processes of how the sighted audio describer perceives and segments the film’s unfolding chain of events and in what way the visually impaired end users conceive the structure, content, and segmentation of such events in relation to the produced AD. There is, however, virtually no research on this interplay in relation to AD. In this study, we scrutinize live AD of a film from two trained audio describers and examine how events are structured, segmented and construed in their AD. Results demonstrate that the event segmentation experienced from the film is indeed a fundamental part of how AD is structured and construed. It was found that AD at event boundaries was highly sensitive to different spatiotemporal circumstances and this relationship depends on semantic resources for expressing AD. Lay summary In our everyday lives, we naturally organize and remember our experiences as meaningful sequences of events. This happens the same way when we watch movies – we see them as a series of events happening in different places, times, and with changing character dynamics. Now, think about making this cinematic experience accessible to people with visual impairments. This is where a sighted translator, called an audio describer, becomes really important. To make movies understandable for a visually impaired audience, a sighted audio describer describes crucial visual events in the film. Their goal is to create a "narrative equivalence" between the original film and the audio-described version. Thus, the main challenges for audio describers are deciding what information to describe, when to do it, and how to articulate it effectively. In the current case study, we looked at two audio descriptions for the Swedish film "Skumtimmen" (English title: "Echoes from the Dead"). These descriptions were created by trained audio describers. We wanted to see how these narrators organized, broke down, and verbalized the course of events in their audio descriptions. To achieve this, we focused on two important parts of storytelling – changes in time and location – and paid special attention to how often and in what way audio describers verbalized these event boundaries. What we found is that both describers indeed verbalized most event boundaries. This shows that these boundaries are a big part of how audio descriptions are put together. We also learned that the descriptions at these event boundaries were very sensitive to when and where key events unfolded in the story. The way these boundaries were described depended on fundamental cognitive and linguistic resources to talk about space and time. Our findings help us understand how breaking down events in a film is a crucial part of shaping audio descriptions. Plus, our research highlights how important it is to study how events are broken down and verbalized, giving useful insights to improve audio descriptions and train future audio describers. The ultimate goal is to make the experience better for visually impaired viewers, contributing to a more inclusive and enjoyable audiovisual world.

Event Boundary Perception in Audio Described Films by People Without Sight

“…Utilizing the event-index model Zwaan & Radvansky, 1998) and its extension to motion pictures (Zacks et al, 2010;Cutting & Iricinschi, 2015), the film had previously undergone systematic coding for coarse-grained event boundaries aligning with EVENT BOUNDARY PERCEPTION IN AD FILMS spatial and temporal alterations, demonstrating high inter-rater reliability from two independent coders (Cohen's kappa = 0.85, p < .001). For further details on the coding procedure, see Holsanova et al (2023). Spatial changes either involved a transition from one location to another within the same time frame (e.g., from the outside to the inside of a house), or a distinct transition within one and the same location setting (e.g., the protagonist moving from one room to another in the same house).…”

“…Previous research on event segmentation underscores the significance of assessing event units across various hierarchical levels, enabling identification at different temporal grains, ranging from fine-grained (brief) to coarse-grained (extended) dimensions (e.g., Kurby & Zacks, 2008;Papenmeier, Maurer, & Huff, 2019;Sebastian, Ghose, & Huff, 2018;Zacks et al, , 2010. The current study does not differentiate among these hierarchical event levels but rather concentrates on the "natural" event segmentation of prominent higher-level spatiotemporal changes, which are anticipated to predominantly correspond with a coarse-grained segmentation level (see Kurby & Zacks, 2008;Papenmeier et al, 2019;Sebastian et al, 2018;Zacks et al, , 2010, and expected to be conveyed in the AD (Holsanova et al, 2023;Vercauteren, 2021;Rai, Greening, & Petré, 2010).…”

“…In the present study, we focus on changes in time and location. This choice is primarily motivated by the salience of temporal and locational changes, which have been central in prior research (e.g., Cutting and Iricinschi, 2015;Speer et al, 2003;Speer & Zacks, 2005; EVENT BOUNDARY PERCEPTION IN AD FILMS al., 2010;Zacks, 2021) and are expected to be conveyed by the audio describer (see Holsanova et al, 2023;Vercauteren, 2021;Rai, Greening, & Petré, 2010). Spatiotemporal settings constitute fundamental narrative building blocks and are intrinsically connected to characters and their actions.…”

“…Despite the limited extent of research in this field, earlier investigations have contributed valuable insights. Prior research has studied event segmentation in AD production, emphasizing a significant overlap between event boundaries and the resultant AD (Holsanova et al, 2023). However, considerable variability was noted in the expression of event boundaries among individual audio describers, both within and across instances.…”

“…Notably, while previous research has recognized the importance of event segmentation in relation to AD production (Holsanova, Blomberg, Blomberg, Gärdenfors, & Johansson, 2023), prior investigations have not examined how people with a visual impairment perceive event segmentation when receiving the AD.…”