Confirming Statements about Pictures of Natural Scenes: Evidence of the Processing of Gist from Eye Movements

Abstract

Combined displays of graphics and text, such as figure captions in newspapers and books, lead to distinctive inspection patterns, or scanpaths. Readers characteristically look very briefly at the picture, and then read the caption, and then look again at the picture. The initial inspection of the picture is the focus of interest in the present experiment, in which we attempted to modify the inspection by giving participants advance knowledge of the subject of a sentence (the cued object) that was to be verified or denied on the basis of whether it correctly described some aspect of the scene shown in the picture. Eye fixations were recorded while the viewers looked at the picture and the sentence in whatever sequence they chose. By allowing viewers to know the subject of the sentence in advance, we asked whether patterns of fixations on the sentence and on the second inspection of the picture would reflect prior knowledge of the focus of the sentence. Providing advance information did not influence eye movements while reading the sentence. It did, however, increase the number of fixations in the initial inspection of the picture, and it also reduced the number and duration of the fixations on the pictures overall. The results suggest that cueing participants to the object allowed increased coding in the initial inspection of the picture, though the benefit of such coding only becomes apparent when the picture is inspected for the second time.

Eye movements when looking at print advertisements: the goal of the viewer matters

“…Rather, viewers tend to go to that part of the ad that is maximally informative for their goals and then they tend to either scan the picture (as in the present study) or read the text (as in Rayner et al) depending on what is most functional for their goals. Although there is relatively little information on the complex processes involved in integrating text and pictures (though see Carroll et al, 1992; Hegarty, 1992; Rayner et al, 2001; Underwood and Foulsham, 2006; Underwood et al, 2004, 2005) across the studies that do exist (including the present study), the evidence seems to be that viewers tend to sample either the picture or the ad quite efficiently and that they do not move their eyes back and forth between the two parts of the array. Thus, depending on the task, it appears that viewers sample the pictorial information and then use the text for confirmation, or they read the text and then sample the pictorial information for confirmation.…”

“…More recent work by Hayhoe and Ballard (2005), Land and Hayhoe (2001), and Underwood and Foulsham (2006) has also clearly demonstrated that the goal of the viewer influences eye fixation times and eye-movement patterns. With respect to the second issue, while there is a considerable amount of research on eye movements while reading and while looking at scenes (Rayner, 1998; Rayner & Castelhano, 2007), there is relatively little eye-movement data available regarding the complex processes involved in integrating text and pictures (see Carroll, Young, & Guertin, 1992; Hegarty, 1992; Underwood, Crundall, & Hodson, 2005; Underwood, Jebbett, & Roberts, 2004 for some notable exceptions). We will return to discuss this issue of how text and pictures are integrated in the Section ‘General Discussion’, along with the implications of the present results for how people look at ads and distribute their attention to critical parts of print ads.…”

Processing Spatial Relations With Different Apertures of Attention

“…Similarly, Carlson and Van Deman (2004) have proposed that during the apprehension of a categorical spatial relation (e.g., the coffee mug is ''below'' the coffee pot), spatial attention is moved from the reference object (the coffee pot) in the direction of a target or located object (the mug). Research on eye movements (i.e., on overt, top-down, attention), while participants verify verbal descriptions (containing locative prepositions) of a scene, confirms the presence of systematic, serial, fixations onto the objects identified by the sentence (e.g., Tanenhaus, Spivey-Knowlton, Eberhard, & Sedivy, 1995;Underwood, Crundall, & Hodson, 2005). Thus, within this account, the ''windowing of attention'' process may be sequential in nature, having an initial and final windowing where different spatial regions are highlighted in series.…”

The effect of the first glimpse at a scene on eye movements during search

“…The term "gist" has been used to refer to both the conceptually driven (Underwood, Crundall, & Hodson, 2005) and perceptually driven global visual properties of the scene (Greene & Oliva, 2009;Torralba, Oliva, Castelhano & Henderson, 2006). Global visual properties include the spatial layout information and the statistical regularities of the visual properties.…”