Reconsidering Yarbus: A failure to predict observers’ task from eye movement patterns

Abstract

In 1967, Yarbus presented qualitative data from one observer showing that the patterns of eye movements were dramatically affected by an observer's task, suggesting that complex mental states could be inferred from scan paths. The strong claim of this very influential finding has an never been rigorously tested. Our observers viewed photographs for 10 seconds each. They performed one of four image-based tasks while eye movements were recorded. A pattern classifier, given features from the static scan paths, could identify the image and the observer at above-chance levels. However, it could not predict a viewer's task. Shorter and longer (60 sec) viewing epochs produced similar results. Critically, human judges also failed to identify the tasks performed by the observers based on the static scan paths. The Yarbus finding is evocative, and while it is possible an observer's mental state might be decoded from some aspect of eye movements, static scan paths alone do not appear to be adequate to infer complex mental states of an observer.

Analysis of Scores, Datasets, and Models in Visual Saliency Prediction

“…Future extensions would be designing standard and more challenging decoding scenarios and considering other features such as saccade sequence (scanpath) information for better discriminating tasks, populations of patients, or stimulus category. Our results are particularly important, if we notice that a recent study [57] reported a failure in decoding subject's task from fixations. Another promising research direction is designing better saliency evaluation scores which: (1) are able to better distinguish fixated vs. non-fixated locations, and (2) are able to discount confounding parameters such as center-bias.…”

What Am I Looking at? Interpreting Dynamic and Static Gaze Displays

“…(2012), who suggested that observers were unable to infer a person's task based on a static gaze display. Possibly, this is due to the fact that our instructions were a selection of three of Yarbus’s (1967) original instructions, one of which we could reasonably expect to stand out from the other two (i.e., “objects”), whereas we could expect the other two to partially overlap in terms of the areas fixated, but not in the order of fixations (i.e., “ages” and “away”; DeAngelus & Pelz, 2009).…”

Predicting an observer's task using multi-fixation pattern analysis

“…This means that using an off-the-shelf classification algorithm requires turning this time-series data into a single vector. Greene et al [2012] do this by generating a single vector of summary statistics per trial. We used a similar approach.…”

“…In a study similar to Yarbus's [1967], Greene et al [2012] attempted to use MFPA to infer their subject's tasks. In their experiment, the subjects' eye movements were recorded while photographs were shown to them.…”