Deep-BCN: Deep Networks Meet Biased Competition to Create a Brain-Inspired Model of Attention Control

Adeli, Hossein; Zelinsky, Gregory J.

doi:10.1109/cvprw.2018.00259

Cited by 16 publications

(12 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, human defined objects override low-level features (Stoll, Thrun, Nuthmann, & Einhäuser, 2015). Given the recent success of image-computable models that explicitly or implicitly incorporate object content in predicting fixation probability (Huang, Shen, Boix, & Zhao, 2015;Kümmerer, Wallis, & Bethge, 2016) or scanpaths (Adeli & Zelinsky, 2018), however, the differentiation between high-level or semantic content on the one hand and low-level features on the other hand might eventually become void, and should be replaced by a notion of image-computability. Extending such models, which currently have a strong focus on fixation probability, to predict fixation durations will be an interesting issue for future research.…”

Section: Discussionmentioning

confidence: 99%

Fixation durations in natural scene viewing are guided by peripheral scene content

2020

View full text Add to dashboard Cite

Fixation durations provide insights into processing demands. We investigated factors controlling fixation durations during scene viewing in two experiments. In Experiment 1, we tested the degree to which fixation durations adapt to global scene processing difficulty by manipulating the contrast (from original contrast to isoluminant) and saturation (original vs. grayscale) of the entire scene. We observed longer fixation durations for lower levels of contrast, and longer fixation durations for grayscale than for color scenes. Thus fixation durations were globally slowed as visual information became more and more degraded, making scene processing increasingly more difficult. In Experiment 2, we investigated two possible sources for this slowdown. We used "checkerboard" stimuli in which unmodified patches alternated with patches from which luminance information had been removed (isoluminant patches). Fixation durations showed an inverted immediacy effect (longer, rather than shorter, fixation durations on unmodified patches) along with a parafoveal-on-foveal effect (shorter fixation durations, when an unmodified patch was fixated next). This effect was stronger when the currently fixated patch was isoluminant as opposed to unmodified. Our results suggest that peripheral scene information substantially affects fixation durations and are consistent with the notion of competition among the current and potential future fixation locations.

show abstract

Section: Discussionmentioning

confidence: 99%

Fixation durations in natural scene viewing are guided by peripheral scene content

2020

View full text Add to dashboard Cite

show abstract

“…This cost is arguably an evolutionary force behind the biological machinery used to implement eye movements and eccentricitydependent sampling as done in Ref. 125. Consistent with this idea, Eckstein et al 132 have shown that, unlike current architectures for object localization that scan for objects exhaustively across scales, human search is largely guided by context.…”

Section: Attention and Searchmentioning

confidence: 92%

“…Related recent work by Adeli and Zelinsky provided a biologically inspired implementation of biased competition theory, whereby the multiple objects in a display compete with each other for attention and a top‐down signal is used to disambiguate and bias this competition in favor of the sought target 125 . Such feature‐based modulation is more efficient when applied at later stages of the visual hierarchy, 124,126 which is consistent with physiological observations showing that both spatial and feature‐based attention is considerably weaker in early visual cortical areas compared with higher visual cortical areas.…”

Section: The Role Of Recurrence Beyond Recognitionmentioning

confidence: 99%

Beyond the feedforward sweep: feedback computations in the visual cortex

Kreiman

Serre

2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

Visual perception involves the rapid formation of a coarse image representation at the onset of visual processing, which is iteratively refined by late computational processes. These early versus late time windows approximately map onto feedforward and feedback processes, respectively. State‐of‐the‐art convolutional neural networks, the main engine behind recent machine vision successes, are feedforward architectures. Their successes and limitations provide critical information regarding which visual tasks can be solved by purely feedforward processes and which require feedback mechanisms. We provide an overview of recent work in cognitive neuroscience and machine vision that highlights the possible role of feedback processes for both visual recognition and beyond. We conclude by discussing important open questions for future research.

show abstract

“…When searching for objects in scenes, scene context can guide attention to the target. To date, there have been very few deep network models attempting to predict human search fixations [2,51,58]. What all of these models have in common, however, is that they use some algorithm, and knowledge about a particular source of information (target features, meaning, context, etc), to prioritize image locations for fixation selection.…”

Section: Introductionmentioning

confidence: 99%

Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning

Yang

Huang

Chen

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Being able to predict human gaze behavior has obvious importance for behavioral vision and for computer vision applications. Most models have mainly focused on predicting free-viewing behavior using saliency maps, but these predictions do not generalize to goal-directed behavior, such as when a person searches for a visual target object. We propose the first inverse reinforcement learning (IRL) model to learn the internal reward function and policy used by humans during visual search. The viewer's internal belief states were modeled as dynamic contextual belief maps of object locations. These maps were learned by IRL and then used to predict behavioral scanpaths for multiple target categories. To train and evaluate our IRL model we created COCO-Search18, which is now the largest dataset of high-quality search fixations in existence. COCO-Search18 has 10 participants searching for each of 18 target-object categories in 6202 images, making about 300,000 goaldirected fixations. When trained and evaluated on COCO-Search18, the IRL model outperformed baseline models in predicting search fixation scanpaths, both in terms of similarity to human search behavior and search efficiency. Finally, reward maps recovered by the IRL model reveal distinctive target-dependent patterns of object prioritization, which we interpret as a learned object context.

show abstract

Deep-BCN: Deep Networks Meet Biased Competition to Create a Brain-Inspired Model of Attention Control

Cited by 16 publications

References 53 publications

Fixation durations in natural scene viewing are guided by peripheral scene content

Fixation durations in natural scene viewing are guided by peripheral scene content

Beyond the feedforward sweep: feedback computations in the visual cortex

Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning

Contact Info

Product

Resources

About