Insights From Mental Model Theory and Cognitive Narratology as a Tool for Content Selection in Audio Description

Abstract

One of the main questions in audio description (AD) to which no systematic answers have been provided yet, is how to decide what information you include in your description and – if there is not enough time to describe everything – how you prioritize that information. In the present paper I want to propose an answer to this problem by asking the question: how do audiences process (filmic) stories and what information do they need to process them? The basic idea underlying this question is that people process and interpret stories by creating mental models (Johnson-Laird, 1983) of these stories. The paper explains how these models are created, what information is necessary to create them and what is optional, thus helping describers to decide what information in their description is “need-to-have” and what is “nice-to-have”. The theoretical explanation will be applied to the opening of the film Slumdog millionaire (Boyle, 2008), to illustrate how the theory works and can be used in daily practice. Lay summary Audio description (AD) for film is a service for people with sight loss that weaves a verbal description of visual elements and unclear sound effects they do not have access to, between the dialogues of the original production. Since this description cannot interfere with the dialogues, there often is very little time for AD and describers will have to decide what to include and what to leave out of their descriptions. In this article, I present a way to tackle this problem, based on the basic idea that films generally tell stories and that the audio description should allow the target audience to recreate that story in their minds. More specifically I focus on two questions, namely a) how do audiences mentally recreate stories and b) what elements do they need to do so. Insights into these two questions will show audio describers what information the target audience needs to recreate the story told in the film, and hence will help them to decide what information they really need to include in their AD. After a theoretical exploration of these two questions, the approach will be illustrated by means of a concrete example, taken from the film Slumdog millionaire (Boyle, 2008).

Event Segmentation in the Audio Description of Films

“…In the present study, we focus on changes in time and location. The main reason for this is that temporal and locational changes are salient categories, which both have been in focus in previous studies (e.g., Speer et al, 2003;Speer & Zacks, 2005;Zacks et al, 2010), and importantly, they are expected to be expressed by the audio describer (see Vercauteren, 2021;Rai et al, 2010). Also, spatiotemporal settings belong to the basic narrative building blocks in storytelling and are intrinsically linked to the story's characters and their actions.…”

“…For instance, spatiotemporal settings can be linked to different characters (e.g., characters that are associated with a particular pub, car, school, or city), or represent different time periods (e.g., flashbacks to a setting in the 60s versus present time). Spatiotemporal settings have been demonstrated to constitute a critical part of AD Vandaele, 2012;Vercauteren, 2021;, and most of the existing guidelines on AD recommend that spatiotemporal features are to be included in the descriptions (cf. Rai et al, 2010;.…”

“…Other challenges include the richness of the visual mode, the need to prioritise information and to select appropriate verbal means for describing the visual content. In essence, when producing AD, the audio describer constantly needs to evaluate what to describe, when to describe it, and how to describe it (Holsanova, 2016;2022;Holsanova et al, 2016;Vercauteren, 2021). The efficacy of this communication is critically dependent upon basic cognitive processes of how the sighted audio describer perceives and segments the film's unfolding chain of events and in what way the visually impaired end users conceive the structure, content, and segmentation of such events in relation to the produced AD and the sound expressions from the film (cf.…”

“…Thus, a fundamental question in successful AD should concern how the sequence of events unfolding in the movie relates to the verbal narration communicated through AD. To our knowledge, no previous studies have investigated event segmentation in AD (but see Vercauteren, 2021, where concepts from event cognition and mental model theory are used to analyse content selection in AD).…”

Event Boundary Perception in Audio Described Films by People Without Sight

“…For instance, it is imperative for the audio describer to refrain from speaking during dialogues and to discern when and how to elucidate specific sounds, such as identifying the source of gunshots (who is shooting at whom) or allowing sounds to convey their meaning independently, as in the case of a door knocking or a ringing telephone. In essence, the audio describer must continually assess what to describe, when to provide descriptions, and how to articulate them (Holsanova, 2016;2022;Vercauteren, 2021). The effectiveness of this form of communication relies heavily on fundamental cognitive processes governing how sighted audio describers perceive and segment the unfolding narrative events within a film, and critically on how receivers with a visual EVENT BOUNDARY PERCEPTION IN AD FILMS impairment perceive and conceptualize the structure, content, and segmentation of these events in relation to the provided AD (Holsanova, 2016;2022;Vandaele, 2012).…”