Towards the quantitative evaluation of visual attention models

Bylinskii, Zoya; DeGennaro, Ellen M; Rajalingham, Rishi; Ruda, Harald; Zhang, J.; Tsotsos, John K.

doi:10.1016/j.visres.2015.04.007

Cited by 53 publications

(54 citation statements)

References 147 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Current eye tracking experimentation represent indicators of saliency as the probability of fixations on certain 274 regions of an image 2 . Metrics used in saliency benchmarks [40] consider all fixations during viewing time with same 275 importance, making saliency hypotheses unclear of which computational procedures perform best using real image 276 datasets. Previous psychophysical studies [16,17] revealed that fixations guided by bottom-up attention are influenced 277 by the type of features that appear in the scene and their relative feature contrast.…”

Section: Model Evaluation 273mentioning

confidence: 99%

“…39 Therefore, modelization of attention should consider as well the influences of task and many other top-down effects. 40 41 Initial hypotheses by Li [24,25] suggested that visual saliency is processed by the lateral interactions of V1 cells. 42 In their work, pyramidal cells and interneurons in the primary visual cortex (V1, Brodmann Area 17 or striate cortex) 43 and their horizontal intracortical connections are seen to modulate activity in V1.…”

Section: Introductionmentioning

confidence: 99%

“…Therewith, our model has also been able to reproduce visual discomfort, as relative contrast energy of particular 67 region on a scene is found to produce hyperexcitability in V1 [32,33], one of possible causes of producing certain 68 conditions such as malaise, nausea or even migraine. Previous neurodynamic [34,35,36,37,38,39] and saliency 69 models [8,9,40] have been able to predict eye movements. However, most of these models have been built specifically 70 for visual saliency in a free-viewing task, a characteristic that denies their biological plausibility for modeling distinct 71 visual processing mechanisms or other visual processes simultaneously.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Modeling Bottom-Up and Top-Down Attention with a Neurodynamic Model of V1

Berga

Otazu

2019

Preprint

View full text Add to dashboard Cite

Previous studies in that line suggested that lateral interactions of V1 cells are responsible, among other visual effects, of bottom-up visual attention (alternatively named visual salience or saliency). Our objective is to mimic these connections in the visual system with a neurodynamic network of firing-rate neurons. Early subcortical processes (i.e. retinal and thalamic) are functionally simulated. An implementation of the cortical magnification function is included to define the retinotopical projections towards V1, processing neuronal activity for each distinct view during scene observation. Novel computational definitions of top-down inhibition (in terms of inhibition of return and selection mechanisms), are also proposed to predict attention in Free-Viewing and Visual Search conditions. Results show that our model outpeforms other biologically-inpired models of saliency prediction as well as to predict visual saccade sequences during free viewing. We also show how temporal and spatial characteristics of inhibition of return can improve prediction of saccades, as well as how distinct search strategies (in terms of feature-selective or category-specific inhibition) predict attention at distinct image contexts. Author summarySaliency maps are the representations of how certain visual regions attract attention in a visual scene, and these can be measured with eye movements. A myriad of computational models with artificial and biological inspiration have been able to acquire outstanding predictions of human fixations. However, most of these models have been built specifically for visual saliency, a characteristic that denies their biological plausibility for modeling distinct visual processing mechanisms or other visual processes simultaneously. In addition to saliency, our approach is also able to efficiently work for other tasks (without applying any type of training or optimization and keeping the same parametrization) such as Visual Search, Visual Discomfort [1], Brightness [2] and Color Induction [3]. By performing simulations of human physiology and its mechanisms, we propose to build a unified model that could be extended to predict and understand distinct perceptual processes in which V1 is responsible.2 discriminate redundant information [4][5][6]. In order to filter or select the information to 3 May 3, 2019 1/32 be processed in higher areas of visual processing in the brain, the HVS guides eye 4 movements towards regions that appear to be visually conspicuous or distinct in the 5 scene. This phenomena was observed during visual search tasks [7, 8], where detecting 6 early visual features (such as orientation, color or size) was done in parallel 7 (pre-attentively) or required either a serial "binding" step depending on scene context. 8 Koch & Ullman [9] came up with the hypothesis that neuronal mechanisms involved in 9 selective visual attention generate a unique "master" map from visual scenes, coined 10 with the term "saliency map". From that, Itti, Koch & Ullman [10] presented a 11 computational i...

show abstract

Section: Model Evaluation 273mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Modeling Bottom-Up and Top-Down Attention with a Neurodynamic Model of V1

Berga

Otazu

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…This is why, in order to compare the saliency models inside each category, some characteristics have been added into the descriptive sheets. However, as explained in [44], the diversity of models makes taxonomy and comparison in the field of visual attention particularly difficult. The purpose of this section is to provide readers with a global view of each model characteristic.…”

Section: Conclusion: a Taxonomy Of The Algorithmsmentioning

confidence: 99%

Bottom-Up Saliency Models for Still Images: A Practical Review

Riche

Mancaş

2016

From Human Attention to Computational Attention

View full text Add to dashboard Cite

There is an increasing interest to utilize human visual attention abilities on computational systems. This is especially the case for computer vision which needs to select the most relevant parts within a large amount of data. Therefore, modeling visual attention, particularly the bottom-up part, has been a very active research area over the past 20 years. Many different models of visual bottom-up attention are now available online. They take as input natural images and output a saliency map which gives the probability of each pixels to grab our attention. In this chapter, a state of the art of saliency-based models has been done. Those models will be used in the next chapters for explaining the validation of saliency models in computer science. BackgroundThe taxonomy proposed in this section is very simple and based on the historical development of saliency models. It is the most efficient one to present the study and to validate the saliency models which will be detailed in the next chapters. We distinguish two big classes of models, corresponding to different types of outputs.Chronologically, the first algorithm type is mostly inspired from the psychological and neurobiological theories. It uses eye tracking data (fixation map) as ground truth. This is why we call models corresponding to this type eye tracking (ET)-based models in the present chapter. The purpose of this class of models is to N. Riche ( ) • M. Mancas

show abstract

“…This architecture has been the main inspiration for current saliency models [62,43], that alternatively use distinct mechanisms (accounting for different levels of processing, context or tuning depending on the scene) but preserving same or similar structure for these steps. Although current state-of-the-art models precisely resemble eye-tracking fixation data [6,9], we question if these models represent saliency. We will test this hypothesis with a novel synthetic image dataset.…”

Section: Introductionmentioning

confidence: 98%

SID4VAM: A Benchmark Dataset With Synthetic Images for Visual Attention Modeling

Berga

Vidal

Otazu

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

A benchmark of saliency models performance with a synthetic image dataset is provided. Model performance is evaluated through saliency metrics as well as the influence of model inspiration and consistency with human psychophysics. SID4VAM is composed of 230 synthetic images, with known salient regions. Images were generated with 15 distinct types of low-level features (e.g. orientation, brightness, color, size...) with a target-distractor popout type of synthetic patterns. We have used Free-Viewing and Visual Search task instructions and 7 feature contrasts for each feature category. Our study reveals that state-ofthe-art Deep Learning saliency models do not perform well with synthetic pattern images, instead, models with Spectral/Fourier inspiration outperform others in saliency metrics and are more consistent with human psychophysical experimentation. This study proposes a new way to evaluate saliency models in the forthcoming literature, accounting for synthetic images with uniquely low-level feature contexts, distinct from previous eye tracking image datasets.

show abstract

Towards the quantitative evaluation of visual attention models

Cited by 53 publications

References 147 publications

Modeling Bottom-Up and Top-Down Attention with a Neurodynamic Model of V1

Modeling Bottom-Up and Top-Down Attention with a Neurodynamic Model of V1

Bottom-Up Saliency Models for Still Images: A Practical Review

SID4VAM: A Benchmark Dataset With Synthetic Images for Visual Attention Modeling

Contact Info

Product

Resources

About