Real and predicted influence of image manipulations on eye movements during scene recognition

Abstract

In this paper, we investigate how controlled changes to image properties and orientation affect eye movements for repeated viewings of images of natural scenes. We make changes to images by manipulating low-level image content (such as luminance or chromaticity) and/or inverting the image. We measure the effects of these manipulations on human scanpaths (the spatial and chronological path of fixations), additionally comparing these effects to those predicted by a widely used saliency model (L. Itti & C. Koch, 2000). Firstly we find that repeated viewing of a natural image does not significantly modify the previously known repeatability (S. A. Brandt & L. W. Stark, 1997; D. Noton & L. Stark, 1971) of scanpaths. Secondly we find that manipulating image features does not necessarily change the repeatability of scanpaths, but the removal of luminance information has a measurable effect. We also find that image inversion appears to affect scene perception and recognition and may alter fixation selection (although we only find an effect on scanpaths with the additional removal of luminance information). Additionally we confirm that visual saliency as defined by L. Itti and C. Koch's (2000) model is a poor predictor of real observer scanpaths and does not predict the small effects of our image manipulations on scanpaths.

Recurrence quantification analysis of eye movements

“…The number of transformation steps can thus be used as a measure of the distance between the scan paths. Scan path comparisons based on string-edit distance have been used successfully by many researchersfor example, , Underwood, Foulsham, and Humphrey (2009), Foulsham and Kingstone (in press), and Harding and Bloj (2010). A second common scan path comparison is a linear distance algorithm that compares fixation sequences on the basis of the average distances between fixations (Henderson, Brockmole, Castelhano, & Mack, 2007;Mannan, Ruddock, & Wooding, 1995).…”

iMap: a novel method for statistical fixation mapping of eye movement data

“…As in previous approaches (e.g., Barrington et al, 2008;Bruce & Tsotsos, 2009;Buchan et al, 2007;Harding & Bloj, 2010;Henderson, 2003;Kita et al, 2010;Pomplun et al, 1996;Tatler et al, 2010; Torralba et al, 2006;Wooding, 2002), this method does not rely on the subjective definition of ROIs; in fact, it simply does not require the use of ROIs. Crucially, however, iMap offers some advantages relative to previous methods.…”

“…It is worth noting that methods sharing similarities with iMap have been introduced by various authors before us (e.g., Barrington, Marks, Hsiao, & Cottrell, 2008;Bruce & Tsotsos, 2009;Buchan, Paré, & Munhall, 2007;Harding & Bloj, 2010;Henderson, 2003;Kita et al, 2010;Pomplun, Ritter, & Velichkovsky, 1996;Tatler, Wade, Kwan, Findlay, & Velichkovsky, 2010;Torralba, Oliva, Castelhano, & Henderson, 2006;Wooding, 2002). Here, we briefly present the characteristics that few of these methods share with iMap and what differentiate them from it.…”

“…Note that using a Gaussian smoothing on fixations, weighted by their durations, is not novel per se, and heat map representations have become very popular in the last few years (see, e.g., Barrington et al, 2008;Bruce & Tsotsos, 2009;Buchan et al, 2007;Harding & Bloj, 2010;Henderson, 2003;Kita et al, 2010;Tatler et al, 2010;Torralba et al, 2006;Watanabe, Matsuda, Nishioka, & Namatame, 2011;Wooding, 2002). However, in most of these cases, the heat maps were used only for illustration purposes.…”

“…We would particularly like to mention the Scanmatch toolbox from Cristino, Mathôt, Theeuwes, and Gilchrist (2010), the scan path similarity measure of Jarodzka, Holmqvist, and Nyström (2010), and the approach used by Mannan, Kennard, and Husain (2009 Harding & Bloj, 2010, for a use of this method). A key advantage of Cristino et al's method is that it is provided in an implemented, freely available toolbox, allowing researchers who are not expert in programming to use their approach.…”