LATEST: A model of saccadic decisions in space and time.

Abstract

Many of our actions require visual information and for this it is important to direct the eyes to the right place at the right time. Two or three times every second, we must decide both when and where to direct our gaze. Understanding these decisions can reveal the moment-to-moment information priorities of the visual system and the strategies for information sampling employed by the brain to serve ongoing behaviour. Most theoretical frameworks and models of gaze control assume that the spatial and temporal aspects of fixation point selection depend on different mechanisms. We present a single model that can simultaneously account for both when and where we look. Underpinning this model is the theoretical assertion that each decision to move the eyes is an evaluation of the relative benefit expected from moving the eyes to a new location compared to that expected by continuing to fixate the current target. The eyes move when the evidence that favours moving to a new location outweighs that favouring staying at the present location. Our model provides not only an account of when the eyes move, but also what will be fixated.That is, an analysis of saccade timing alone enables us to predict where people look in a scene. Indeed our model accounts for fixation selection as well as (and often better than) current computational models of fixation selection in scene viewing. Much that we do requires visual information. However, the manner in which our eyes sample the environment greatly limits the information that is available to us: the small window of clear vision at the centre of gaze can only be directed to three locations or so in the environment each second. The valuable resource of high quality vision must therefore be allocated with care in order to provide the right information at the right time. Thus, understanding the mechanisms that underlie fixation allocation in space and time can tell us about the moment-to-moment information priorities of the visual system and the strategies for information sampling employed by the brain to serve ongoing behaviour. A complete understanding of eye movement behaviour must therefore encompass what determines both when and where we look. To date, attempts to decode the mechanisms underlying the temporal and spatial aspects of gaze control have remained separate in the literature. In this paper, we will argue this independence has led to a false dichotomy and instead offer the novel theoretical argument that a single decision process determines when our eyes move as well as where they move.The logic behind our proposal is straightforward yet entirely novel. The purpose of saccades is to acquire information about the outside world, and the outside world is complex, containing many potential sources of information. Questions regarding when and where to move the eyes can therefore be thought of in terms of competition between different potential sources of information in the environment. We will argue that this competition is resolved via a Bayesian-like process, in whic...

The elephant in the room: Inconsistency in scene viewing and representation.

“…The studies reviewed above suggest that some semantic processing must be going on prior to foveal inspection of the object in order for that object to be selected sooner than others. The idea that semantic information may be processed extrafoveally, prior to fixation, and may inform fixation selection has been suggested previously (see Tatler, Brockmole and Carpenter, 2017).…”

Real‐world scene perception in infants: What factors guide attention allocation?

“…The analysis of attention allocation addresses the where , and the fixation duration analysis addresses the when . This distinction is typical in the scene perception literature, as there are studies examining fixation locations or durations, but almost never both; exceptions are papers by Tatler, Brockmole, and Carpenter () and Einhäuser and Nuthmann (). Since the saliency model of Itti and Koch (), many newer saliency models have been proposed (for a review see, Borji et al, ).…”

The intersection between the oculomotor and hippocampal memory systems: empirical developments and clinical implications

“…For instance, processing of information that is inconsistent with prior knowledge (e.g., an octopus in a barnyard) may require additional time; likewise, fixation durations under visual search instructions are shorter than under instructions to memorize the presented scene, owing to different types of cognitive processing required by each task . In contrast to other models that consider predictions of the location and duration of gaze fixations separately, Tatler et al . developed a model that proposes the same underlying process for both where and when the eyes move.…”

“…developed a model that proposes the same underlying process for both where and when the eyes move. Specifically, the authors note that if the purpose of eye movements is to acquire information, then where and when the eyes move can be modeled by understanding the expected benefit in moving the eyes versus remaining in the current location for ongoing and sufficient information extraction …”

“…In considering the findings from their octopus‐in‐the‐barnyard study, Loftus and Mackworth posited that since gist‐level information can be extracted early from a scene, often within a single fixation, the purpose of the subsequent gaze fixations may be to verify whether the presented information is already contained within existing knowledge structures or schemas. On this view, longer durations of gaze fixations would reflect the time required to link or update the relevant schema with the new information (e.g., octopi can live on farms) . That is, greater viewing is directed at the areas of interest within the environment that are not well represented within, or violate the expectations from, a viewer's schemas, and this occurs for the purpose of continually forming new memories and updating knowledge structures.…”