Saliency Benchmarking Made Easy: Separating Models, Maps and Metrics

Kümmerer, Matthias; Wallis, Thomas S. A.; Bethge, Matthias

doi:10.1007/978-3-030-01270-0_47

Cited by 84 publications

(79 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Active research is ongoing to understand the pros and cons of the saliency measures (e.g. [50]). Many of the current saliency methods compete closely with one another at the top of the existing benchmarks and performances vary in a narrow band (See Figures 3 & 4).…”

Section: Discussion and Outlookmentioning

confidence: 99%

Bottom-Up Attention, Models of

Borji¹,

Tavakoli²,

Bylinskii³

2019

Encyclopedia of Computational Neuroscience

View full text Add to dashboard Cite

DefinitionAttention -a general concept covering all factors that influence selection mechanisms, whether they are scene-driven and bottom-up, or expectation-driven and top-down.Salience -parts of a stimulus (e.g. image, video, audio) that appear to an observer to stand out, relative to their neighboring parts. It is a subjective perceptual quality.Gaze -a coordinated motion of the eyes and the head that is long enough to concentrate on something. It is a key property of attention in natural behavior.Scene free-viewing -A task in which participants are asked to look at an image, without any specific instruction. As a default scene analysis task, the free viewing paradigm offers a wealth of insights regarding the cues that attract attention. Detailed Description History, scope, and organizationDeciphering the computational mechanisms by which the brain deals with the computational complexity of an overwhelmingly high volume of incoming data (at a rate of 10 8 bits/s [46]) and how the brain programs eye movements continue to be important problems in neuroscience. Where humans look in images and videos provides important clues regarding how they perceive static (still images) and dynamic scenes (videos), locate the main focus of the image, recognize actions or events, and identify the main participants.A significant amount of behavioral and computational research on attention has revealed that attention is deployed in two ways: bottom-up (BU) and top-down (TD). The bottom-up component of attention, a.k.a endogenous or stimulus-driven, processes sensory information primarily in a feed-forward manner. Typically, a series of successive transformations are applied to the input visual signal to highlight the most interesting, important, conspicuous, or so-called salient regions [45,36]. In contrast, in top-down attention, a.k.a context-driven, or goal-driven, information related to the ongoing behavior, task, or goal is selected (e.g. staying within the road lanes while driving [52]). The reader is referred to [51,2,27,57,11] for reviews of top-down attention studies. Please see also the chapter on top-down attention in this encyclopedia [66].Conventionally, bottom-up attention models generate a 2D topographic saliency map, where a value at every location determines how salient that location is, relative to its neighbors. The goal in saliency *

show abstract

Section: Discussion and Outlookmentioning

confidence: 99%

Bottom-Up Attention, Models of

Borji¹,

Tavakoli²,

Bylinskii³

2019

Encyclopedia of Computational Neuroscience

View full text Add to dashboard Cite

show abstract

“…Once a model detects the main salient regions in an image, it is necessary to validate its performance over ground‐truth data. There are several metrics commonly used in this field and standardized so different models can be compared, although consistent results cannot always be obtained . Depending on the application and the kind of data used for validation, some metrics can be more appropriate than others.…”

Section: Methodsmentioning

confidence: 99%

“…There are several metrics commonly used in this field and standardized so different models can be compared, although consistent results cannot always be obtained. 34 Depending on the application and the kind of data used for validation, some metrics can be more appropriate than others. We decided to use the following three metrics for our experiment:…”

Section: Validationmentioning

confidence: 99%

Improving unsupervised saliency detection by migrating from RGB to multispectral images

Martínez

Etchebehere²,

Valero

et al. 2019

Color Research & Application

View full text Add to dashboard Cite

Saliency detection has been an important topic during the last decade. The main goal of saliency detection models is to detect the most relevant objects in a given scene. Most of these models use RGB (Red, Green, Blue) images as an input because they mainly focus on applications where features (eg, faces, textures, colors, or human silhouettes) are extracted from color images, and there are many labeled databases available for RGB‐based saliency data. Nevertheless, the use of RGB inputs clearly limits the amount of information from where to extract the salient regions as spectral information is lost during the color image recording. On the contrary, multispectral systems are able to capture more than three bands in a single capture and can retrieve information from the full spectrum at a pixel. The main aim of this study is to investigate the advantages of using multispectral images instead of RGB images for saliency detection within the framework of unsupervised models. We compare the performance of several unsupervised saliency models with both RGB and multispectral images using a specific dataset of multispectral images with ground‐truth data extracted from observers' fixation patterns. Our results show a general improvement when multispectral information is taken into account. The saliency maps estimated by using the multispectral features are closer to the ground‐truth data, with the simplest Graph‐based visual saliency and Boolean Map‐based models showing good relative gain compared with other approaches.

show abstract

“…neural and behavioral models (e.g. [19,8,9,12]). Since 2010 Matthias Bethge has been the director of the Bernstein center and since 2016 vice chair of the national Bernstein network.…”

Section: • Design and Implement Submission And Evaluation Backend • Advmentioning

confidence: 99%

Adversarial Vision Challenge

Brendel

Rauber

Kurakin

et al. 2019

The Springer Series on Challenges in Machine Learning

Self Cite

View full text Add to dashboard Cite

The NIPS 2018 Adversarial Vision Challenge is a competition to facilitate measurable progress towards robust machine vision models and more generally applicable adversarial attacks. This document is an updated version of our competition proposal that was accepted in the competition track of 32nd Conference on Neural Information Processing Systems (NIPS 2018). NIPS 2017 Competition on adversarial attacks and defenses.Co-organised by Alexey Kurakin. This competition pitted models against attacks but only indirectly: attacks were unable to query the models and hence had to device generic adversarial examples that would work against as many models as possible. Devising defenses against such unspecific transferbased attacks is much simpler than becoming robust against model-specific attacks.

show abstract

Saliency Benchmarking Made Easy: Separating Models, Maps and Metrics

Cited by 84 publications

References 51 publications

Bottom-Up Attention, Models of

Bottom-Up Attention, Models of

Improving unsupervised saliency detection by migrating from RGB to multispectral images

Adversarial Vision Challenge

Contact Info

Product

Resources

About