Understanding Low- and High-Level Contributions to Fixation Prediction

Kümmerer, Matthias; Wallis, Thomas S. A.; Gatys, Leon A.; Bethge, Matthias

doi:10.1109/iccv.2017.513

Cited by 260 publications

(220 citation statements)

References 34 publications

Supporting

Mentioning

213

Contrasting

Unclassified

Order By: Relevance

“…However, it should be noted that another class of model based on learning within deep neural networks (DNNs) has recently been advanced as a competitor to traditional saliency models (Vig, Dorr, & Cox, 2014). For example, DeepGaze II, the current top performer in this class, learns where people attend in scenes from training sets of fixations over object features and then predicts fixations on new scenes (Kummerer, Wallis, Gatys, & Bethge, 2017).…”

Section: Limitations and Future Directionsmentioning

confidence: 99%

Meaning Guides Attention in Real-World Scene Images: Evidence from Eye Movements and Meaning Maps

Henderson

Hayes

2017

Preprint

View full text Add to dashboard Cite

We compared the influences of meaning and salience on attentional guidance in scenes. Meaning was captured by "meaning maps" representing the spatial distribution of semantic information in scenes. Meaning maps were coded in a format that could be directly compared to maps of image salience generated from image features. We investigated the degree to which meaning versus image salience predicted human viewers' spatial distribution of attention over scenes, with attention operationalized as duration-weighted fixation density. The results showed that both meaning and salience predicted the distribution of attention, but that when the correlation between meaning and salience was statistically controlled, meaning accounted for unique variance in attention but salience did not. This pattern was observed for early as well as late fixations, for fixations following short as well as long saccades, and for fixations including or excluding the centers of the scenes. The results strongly suggest that meaning guides attention in real world scenes. We discuss the results from the perspective of the cognitive relevance theory of attentional guidance in scenes.

show abstract

Section: Limitations and Future Directionsmentioning

confidence: 99%

Meaning Guides Attention in Real-World Scene Images: Evidence from Eye Movements and Meaning Maps

Henderson

Hayes

2017

Preprint

View full text Add to dashboard Cite

show abstract

“…However, while these models work relatively well for impoverished stimuli, human gaze behaviour towards richer scenes can be predicted at least as well by the locations of objects 16 and perceived meaning 9 . When sematic object properties are taken into account, their weight for gaze prediction far exceeds that of low-level attributes 8,17 . A common thread of low-and high-level salience models is that they interpret salience as a property of the image and treat inter-individual differences as unpredictable 7,18 , often using them as a 'noise ceiling' for model evaluations 18 .…”

Section: Introductionmentioning

confidence: 99%

Individual differences in visual salience vary along semantic dimensions

Haas

Iakovidis

Schwarzkopf

et al. 2018

Preprint

View full text Add to dashboard Cite

150 words; Main text: 2615 words (excl. Methods, fig. and table captions); Methods: 17 1526 words; 5 figures:1 table; 1 supplementary figure 18 19 20 21 22 23 features and task demands govern fixation behaviour, while differences between observers 24 are 'noise'. Here, we investigated the fixations of > 100 human adults freely viewing a large 25 set of complex scenes. We found systematic individual differences in fixation frequencies 26 along six semantic stimulus dimensions. These differences were large (> twofold) and 27 highly stable across images and time. Surprisingly, they also held for first fixations directed 28 towards each image, commonly interpreted as 'bottom-up' visual salience. Their 29 perceptual relevance was documented by a correlation between individual face salience and 30 recognition skills. The dimensions of individual salience and their covariance pattern 31 replicated across samples from three different countries, suggesting they reflect 32 fundamental biological mechanisms of attention. Our findings show stable individual 33 salience differences along semantic dimensions, with meaningful perceptual implications. 34 Salience reflects features of the observer as well as the image. 35 36 image discontinuities of low level attributes, such as luminance, colour and orientation 13 . These 45 low-level models are inspired by 'early' visual neurons and their output correlates with neural 46 responses in subcortical 14 and cortical 15 areas thought to represent neural 'salience maps'. 47However, while these models work relatively well for impoverished stimuli, human gaze 48 behaviour towards richer scenes can be predicted at least as well by the locations of objects 16 and 49 perceived meaning 9 . When sematic object properties are taken into account, their weight for gaze 50 prediction far exceeds that of low-level attributes 8,17 . A common thread of low-and high-level 51 salience models is that they interpret salience as a property of the image and treat inter-individual 52 differences as unpredictable 7,18 , often using them as a 'noise ceiling' for model evaluations 18 . 53 However, even the earliest studies of fixation behaviour noted considerable individual 54 differences 19,20 and basic occulomotor traits, like mean saccadic amplitude and velocity, reliably 55 vary between observers 21-27 . Moreover, recent twin-studies revealed that social attention and 56 gaze traces across complex scenes are highly heritable 28,29 . This suggests individual differences 57 in fixation behaviour are not random, but systematic. However, it is largely unclear, how 58 individuals differ in their fixation behaviour and what may explain these differences. Can 59 individual fixation behaviour be captured along a limited set of dimensions? 60Here, we tested the hypothesis that individual gaze reflects individual salience differences 61 along a limited number of semantic dimensions. We investigated the fixation behaviour of > 100 62 human adults freely viewing 700 complex scenes, containing thousands of semantica...

show abstract

“…Rather than simply remapping the color histogram or normalizing an image for nearby luminance, automatic mechanisms are thought to depend on factors such as co-linearity, co-planarity, junctions, feature grouping, and transparency issues such as smoke and rain (Zucker et al 1988;Anderson 1997;Adelson 2000). Biologically driven models are increasingly capable of explaining visual illusions (Blakeslee and McCourt 2004;Li 2011) and predicting gaze patterns based on saliency (Borji 2018;Kummerer et al 2017), but they are data-limited to SDR images and require HDR experimentation to extend their generalizability to real-world vision.…”

Section: Fig 1 A) Two Examples Of Hdr Luminance In Naturalistic Scenmentioning

confidence: 99%

An 11-bit High Dynamic Range (HDR) Luminance Display and Its Use to Discover Contextual Mechanisms of Real-World Luminance Normalization for Visual Acuity and Target Discrimination

Hung

Callahan-Flintoft

Fedele

et al. 2019

Preprint

View full text Add to dashboard Cite

Approved for public release; distribution is unlimited.ii REPORT DOCUMENTATION PAGE Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

show abstract

Understanding Low- and High-Level Contributions to Fixation Prediction

Cited by 260 publications

References 34 publications

Meaning Guides Attention in Real-World Scene Images: Evidence from Eye Movements and Meaning Maps

Meaning Guides Attention in Real-World Scene Images: Evidence from Eye Movements and Meaning Maps

Individual differences in visual salience vary along semantic dimensions

An 11-bit High Dynamic Range (HDR) Luminance Display and Its Use to Discover Contextual Mechanisms of Real-World Luminance Normalization for Visual Acuity and Target Discrimination

Contact Info

Product

Resources

About