Objects predict fixations better than early saliency

Abstract

Humans move their eyes while looking at scenes and pictures. Eye movements correlate with shifts in attention and are thought to be a consequence of optimal resource allocation for high-level tasks such as visual recognition. Models of attention, such as "saliency maps," are often built on the assumption that "early" features (color, contrast, orientation, motion, and so forth) drive attention directly. We explore an alternative hypothesis: Observers attend to "interesting" objects. To test this hypothesis, we measure the eye position of human observers while they inspect photographs of common natural scenes. Our observers perform different tasks: artistic evaluation, analysis of content, and search. Immediately after each presentation, our observers are asked to name objects they saw. Weighted with recall frequency, these objects predict fixations in individual images better than early saliency, irrespective of task. Also, saliency combined with object positions predicts which objects are frequently named. This suggests that early saliency has only an indirect effect on attention, acting through recognized objects. Consequently, rather than treating attention as mere preprocessing step for object recognition, models of both need to be integrated.

Compressed-domain visual saliency models: a comparative study

“…A number of methods have been used to evaluate the accuracy of visual saliency models with respect to gaze point data [9], [10], [17], [35], [36], [54]. Since each method emphasizes a particular aspect of model's performance, to make the evaluation balanced, a collection of methods and metrics is employed in this study.…”

“…It is worth mentioning that instead of using all non-gaze saliency values, these are usually sampled [17], [75]. The idea behind this approach is that an effective saliency model would have higher values at fixation points than at randomly sampled points.…”

Irregularity-based image regions saliency identification and evaluation

“…Some literature judged the results visually from the observers' points of views, which is the easiest method but it depends totally on the feedback from the observers and there is no fixed measure in this method (Hu , et al, 2005), (Einhauser, et al, 2008). Area Under the Curve was used for this purpose in which the saliency map is converted to binary image and then the AUC is calculated and compared to the AUC extracted from the ground truth data (Lin, et al, 2013) (Gide & Karam, 2012), (Zhao & Koch, 2011), (Erdem & Erdem, 2013), and (Kim & Milanfar, 2013).…”

Learning a Saliency Map for Fixation Prediction

“…According to the research in vision [23], objects are better predictors of human fixation than early bottom-up saliency. On the other hand, if the top-down saliency is relatively uniform for an image, human attention is attracted by the bottom-up cues, such as high contrast and vivid color.…”