Simulating human saccadic scanpaths on natural images

Wang, Wei; Chen, Cheng; Wang, Yizhou; Jiang, Tingting; Fang, Fang; Yao, Yuan

doi:10.1109/cvpr.2011.5995423

Cited by 94 publications

(54 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…One solution is parallel implementation of models (e.g., feature extraction on GPU (e.g., [74] [75]). Most models have focused on predicting locations that human observers look at, while few (e.g., [55]) have investigated other aspects of eye fixations, such as saccade dynamics, sequencing (Wang et al [15]), retinal sampling, inhibition of return, the role of context, etc [96]. More work needs to be done in this direction.…”

Section: Discussionmentioning

confidence: 99%

Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study

Borji

Sihite

Itti

2013

IEEE Trans. on Image Process.

549

457

View full text Add to dashboard Cite

Abstract-Visual attention is a process that enables biological and machine vision systems to select the most relevant regions from a scene. Relevance is determined by two components: 1) top-down factors driven by task and 2) bottom-up factors that highlight image regions that are different from their surroundings. The latter are often referred to as "visual saliency". Modeling bottom-up visual saliency has been the subject of numerous research efforts during the past 20 years, with many successful applications in computer vision and robotics. Available models have been tested with different datasets (e.g., synthetic psychological search arrays, natural images or videos) using different evaluation scores (e.g., search slopes, comparison to human eye tracking) and parameter settings. This has made direct comparison of models difficult. Here we perform an exhaustive comparison of 35 state-of-the-art saliency models over 54 challenging synthetic patterns, 3 natural image datasets, and 2 video datasets, using 3 evaluation scores. We find that although model rankings vary, some models consistently perform better. Analysis of datasets reveals that existing datasets are highly center-biased, which influences some of the evaluation scores. Computational complexity analysis shows that some models are very fast, yet yield competitive eye movement prediction accuracy. Different models often have common easy/difficult stimuli. Furthermore, several concerns in visual saliency modeling, eye movement datasets, and evaluation scores are discussed and insights for future work are provided. Our study allows one to assess the state-of-the-art, helps organizing this rapidly growing field, and sets a unified comparison framework for gauging future efforts, similar to the PASCAL VOC challenge in the object recognition and detection domains.

show abstract

Section: Discussionmentioning

confidence: 99%

Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study

Borji

Sihite

Itti

2013

IEEE Trans. on Image Process.

549

457

View full text Add to dashboard Cite

show abstract

“…Such models select the most informative locations and discard redundancies. Wang et al [88] proposed a computational model inspired by information maximization for gaze shifts prediction, which computed three filter responses as a coherent representation for reference sensory responses, fovea periphery resolution discrepancy, and visual working memory. Klein et al [89] introduced a salient object detection method, which has a similar structure to cognitive models, but acknowledges a saliency via information theoretic concept.…”

Section: ) Bayesian Modelsmentioning

confidence: 99%

A Comparison Study of Saliency Models for Fixation Prediction on Infants and Adults

Mahdi

Schlesinger

et al. 2018

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Abstract-Various saliency models have been developed over the years. The performance of saliency models is typically evaluated based on databases of experimentally recorded adult eye fixations. Although studies on infant gaze patterns have attracted much attention recently, saliency based models have not been widely applied for prediction of infant gaze patterns. In this study, we conduct a comprehensive comparison study of eight state-ofthe-art saliency models on predictions of experimentally captured fixations from infants and adults. Seven evaluation metrics are used to evaluate and compare the performance of saliency models. The results demonstrate a consistent performance of saliency models predicting adult fixations over infant fixations in terms of overlap, center fitting, intersection, information loss of approximation, and spatial distance between the distributions of saliency map and fixation map. In saliency and baselines models performance ranking, the results show that GBVS and Itti models are among the top three contenders, infants and adults have bias toward the centers of images, and all models and the center baseline model outperformed the chance baseline model.

show abstract

“…Lastly, we would like to integrate more biologically plausible saccadic models to pilot the gaze exploration of the robot. Indeed, saccades are biased by many factors that do not make the distribution of saccades homogeneous (for example horizontal and vertical saccades are much more common than oblique ones [59]). …”

Section: Discussionmentioning

confidence: 99%

BioVision: A Biomimetics Platform for Intrinsically Motivated Visual Saliency Learning

2019

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Abstract-We present BioVision, a bio-mimetics platform based on the human visual system. BioVision relies on the foveal vision principle based on a set of cameras with wide and narrow fields of view. We present in this platform a mechanism for learning visual saliency in an intrinsically motivated fashion. This model of saliency, learned and improved on-the-fly during the robot's exploration provides an efficient tool for localizing relevant objects within their environment.The proposed approach includes two intertwined components. On the one hand, a method for learning and incrementally updating a model of visual saliency from foveal observations. On the other hand, we investigate an autonomous exploration technique to efficiently learn such a saliency model. The proposed exploration, based on the IAC (Intelligent Adaptive Curiosity) algorithm is able to drive the robot's exploration so that samples selected by the robot are likely to improve the current model of saliency.We then demonstrate that such a saliency model learned directly on a robot outperforms several state-of-the-art saliency techniques, and that IAC can drastically decrease the required time for learning a reliable saliency model. We also investigate the behavior of IAC in a non static environment, and how well this algorithm can adapt to changes.

show abstract

Simulating human saccadic scanpaths on natural images

Cited by 94 publications

References 23 publications

Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study

Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study

A Comparison Study of Saliency Models for Fixation Prediction on Infants and Adults

BioVision: A Biomimetics Platform for Intrinsically Motivated Visual Saliency Learning

Contact Info

Product

Resources

About