Neural Correlates of Fixated Low- and High-level Scene Properties during Active Scene Viewing

Henderson, John M.; Goold, Jessica; Choi, Wonil; Hayes, Taylor R.

doi:10.1162/jocn_a_01599

Cited by 11 publications

(10 citation statements)

References 79 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The primary stimuli consisted of 50 digitized photographs of real-world scenes Hayes, 2017, 2018;Henderson et al, 2020). Sample images are presented in Figures 2 and 3, and all 50 of the images are available in a public repository (https://osf.io/ptsvm/).…”

Section: Methodsmentioning

confidence: 99%

Rapid Extraction of the Spatial Distribution of Physical Saliency and Semantic Informativeness from Natural Scenes in the Human Brain

et al. 2021

Self Cite

View full text Add to dashboard Cite

Physically salient objects are thought to attract attention in natural scenes. However, research has shown that meaning maps, which capture the spatial distribution of semantically informative scene features, trump physical saliency in predicting the pattern of eye moments in natural scene viewing. Meaning maps even predict the fastest eye movements, suggesting that the brain extracts the spatial distribution of potentially meaningful scene regions very rapidly. To test this hypothesis, we applied representational similarity analysis to ERP data. The ERPs were obtained from human participants (N = 32, male and female) who viewed a series of 50 different natural scenes while performing a modified 1-back task. For each scene, we obtained a physical saliency map from a computational model and a meaning map from crowd-sourced ratings. We then used representational similarity analysis to assess the extent to which the representational geometry of physical saliency maps and meaning maps can predict the representational geometry of the neural response (the ERP scalp distribution) at each moment in time following scene onset. We found that a link between physical saliency and the ERPs emerged first (;78 ms after stimulus onset), with a link to semantic informativeness emerging soon afterward (;87 ms after stimulus onset). These findings are in line with previous evidence indicating that saliency is computed rapidly, while also indicating that information related to the spatial distribution of semantically informative scene elements is computed shortly thereafter, early enough to potentially exert an influence on eye movements.

show abstract

Section: Methodsmentioning

confidence: 99%

Rapid Extraction of the Spatial Distribution of Physical Saliency and Semantic Informativeness from Natural Scenes in the Human Brain

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…2020 ). The focus on context-level information measured by saccades was recently found to correspond with activity in higher-order, scene-selective regions ( Henderson et al. 2020 ), demonstrating coarse-level prioritization in combined neuroimaging and eye tracking.…”

Section: Principle 1: Spontaneous Perception Is Constructed By Coarse-to-fine Hierarchical Levels Of Processingmentioning

confidence: 99%

Spontaneous perception: a framework for task-free, self-paced perception

Baror

2021

Neuroscience of Consciousness

View full text Add to dashboard Cite

Flipping through social media feeds, viewing exhibitions in a museum, or walking through the botanical gardens, people consistently choose to engage with and disengage from visual content. Yet, in most laboratory settings, the visual stimuli, their presentation duration, and the task at hand are all controlled by the researcher. Such settings largely overlook the spontaneous nature of human visual experience, in which perception takes place independently from specific task constraints and its time course is determined by the observer as a self-governing agent. Currently, much remains unknown about how spontaneous perceptual experiences unfold in the brain. Are all perceptual categories extracted during spontaneous perception? Does spontaneous perception inherently involve volition? Is spontaneous perception segmented into discrete episodes? How do different neural networks interact over time during spontaneous perception? These questions are imperative to understand our conscious visual experience in daily life. In this article we propose a framework for spontaneous perception. We first define spontaneous perception as a task-free and self-paced experience. We propose that spontaneous perception is guided by four organizing principles that grant it temporal and spatial structures. These principles include coarse-to-fine processing, continuity and segmentation, agency and volition, and associative processing. We provide key suggestions illustrating how these principles may interact with one another in guiding the multifaceted experience of spontaneous perception. We point to testable predictions derived from this framework, including (but not limited to) the roles of the default-mode network and slow cortical potentials in underlying spontaneous perception. We conclude by suggesting several outstanding questions for future research, extending the relevance of this framework to consciousness and spontaneous brain activity. In conclusion, the spontaneous perception framework proposed herein integrates components in human perception and cognition, which have been traditionally studied in isolation, and opens the door to understand how visual perception unfolds in its most natural context.

show abstract

“…It is important to note that this does not necessarily mean that deep saliency models and human ratings of meaning are equivalent 47 . For example, recent neurocognitive work shows human-generated meaning maps produce stronger activation in cortical areas along the ventral visual stream than DeepGaze II 48 . The differences between meaning maps and deep saliency maps are most likely driven by the inherent differences between deep saliency models and human raters.…”

Section: Discussionmentioning

confidence: 99%

Deep saliency models learn low-, mid-, and high-level features to predict scene attention

Hayes

Henderson

2021

Sci Rep

Self Cite

View full text Add to dashboard Cite

Deep saliency models represent the current state-of-the-art for predicting where humans look in real-world scenes. However, for deep saliency models to inform cognitive theories of attention, we need to know how deep saliency models prioritize different scene features to predict where people look. Here we open the black box of three prominent deep saliency models (MSI-Net, DeepGaze II, and SAM-ResNet) using an approach that models the association between attention, deep saliency model output, and low-, mid-, and high-level scene features. Specifically, we measured the association between each deep saliency model and low-level image saliency, mid-level contour symmetry and junctions, and high-level meaning by applying a mixed effects modeling approach to a large eye movement dataset. We found that all three deep saliency models were most strongly associated with high-level and low-level features, but exhibited qualitatively different feature weightings and interaction patterns. These findings suggest that prominent deep saliency models are primarily learning image features associated with high-level scene meaning and low-level image saliency and highlight the importance of moving beyond simply benchmarking performance.

show abstract

Neural Correlates of Fixated Low- and High-level Scene Properties during Active Scene Viewing

Cited by 11 publications

References 79 publications

Rapid Extraction of the Spatial Distribution of Physical Saliency and Semantic Informativeness from Natural Scenes in the Human Brain

Rapid Extraction of the Spatial Distribution of Physical Saliency and Semantic Informativeness from Natural Scenes in the Human Brain

Spontaneous perception: a framework for task-free, self-paced perception

Deep saliency models learn low-, mid-, and high-level features to predict scene attention

Contact Info

Product

Resources

About