Dissociable signatures of visual salience and behavioral relevance across attentional priority maps in human cortex

Abstract

Computational models posit that visual attention is guided by activity within spatial maps that index the image-computable salience and the behavioral relevance of objects in the scene. These spatial maps are theorized to be instantiated as activation patterns across a series of retinotopic visual regions in occipital, parietal, and frontal cortex. Whereas previous research has identified sensitivity to either the behavioral relevance or the image-computable salience of different scene elements, the simultaneous influence of these factors on neural "attentional priority maps" in human cortex is not well understood. We tested the hypothesis that visual salience and behavioral relevance independently impact the activation profile across retinotopically organized cortical regions by quantifying attentional priority maps measured in human brains using functional MRI while participants attended one of two differentially salient stimuli. We found that the topography of activation in priority maps, as reflected in the modulation of region-level patterns of population activity, independently indexed the physical salience and behavioral relevance of each scene element. Moreover, salience strongly impacted activation patterns in early visual areas, whereas later visual areas were dominated by relevance. This suggests that prioritizing spatial locations relies on distributed neural codes containing graded representations of salience and relevance across the visual hierarchy. NEW & NOTEWORTHY We tested a theory which supposes that neural systems represent scene elements according to both their salience and their relevance in a series of "priority maps" by measuring functional MRI activation patterns across human brains and reconstructing spatial maps of the visual scene. We found that different regions indexed either the salience or the relevance of scene items, but not their interaction, suggesting an evolving representation of salience and relevance across different visual areas.

The intersection between the oculomotor and hippocampal memory systems: empirical developments and clinical implications

“…41 The neural instantiation of a priority mapcomposed of the representations that guide where and when the eyes move-is focused on a network of regions that include the lateral intraparietal area (area LIP), 42,43 frontal eye fields (FEFs), 44 and superior colliculus (SC), 45,46 all of which exhibit prioritized representations of visual space and activity that is crucial for the guidance and control of eye movements. [47][48][49][50] A complementary network of regions that includes the dorsolateral prefrontal cortex (DLPFC), anterior cingulate cortex (ACC), and supplementary eye field (SEF) is thought to be involved in the cognitive control of saccades, [51][52][53][54] providing additional goal-directed inputs to the FEF and SC.…”

Inverted encoding models estimate sensible channel responses for sensible models

“…Perhaps most importantly, many studies using IEMs seek to compare channel response profiles, or basis-weighted 'image' reconstructions, across task conditions or timepoints in a trial. As described by Sprague et al (2018), these studies employ a fixed encoding model, such that activation patterns from different conditions are transformed into the same modeled information space, using a single common estimated encoding model (and often that encoding model is estimated using data from a completely different training task, e.g. Sprague et al, 2014Sprague et al, , 2016Sprague et al, , 2018b.…”

“…As described by Sprague et al (2018), these studies employ a fixed encoding model, such that activation patterns from different conditions are transformed into the same modeled information space, using a single common estimated encoding model (and often that encoding model is estimated using data from a completely different training task, e.g. Sprague et al, 2014Sprague et al, , 2016Sprague et al, , 2018b. In this case, the criticisms raised by Liu et al (2018) and Gardner & Liu (2019) do not apply: any arbitrary linear transforms would be applied equivalently to the results from each condition; and differences between conditions would be transformed from participantand stimulus-specific measurement space into the same model-based 'information' space.…”

“…While channel response profiles 1 often look qualitatively similar to neural tuning functions for single units, a point brought up by Gardner & Liu (2019), it is critical to understand that reconstructed channel response profiles cannot be used to draw conclusive inferences about any specific attributes of single-unit response properties (e.g. single unit gain or bandwidth changes, for more on this, see Sprague et al, 2018). Moreover, recovery of peaked channel response profiles does not demonstrate the accuracy (or inaccuracy) of the channel shapes in the encoding model used (Sprague et al, 2018a;Gardner and Liu, 2019).…”

“…We believe the IEM method is most useful when comparing reconstructed channel response profiles across manipulations of stimulus properties (e.g., contrast) or task conditions (e.g., attention), or combinations thereof (Sprague et al, 2018b) using a fixed encoding model across relevant comparisons (Sprague et al, 2018a). When used this way, the criticisms raised by Gardner & Liu (2019) have no substantial bearing on the efficacy of the IEM technique for comparing the impact of experimental manipulations on information represented within aggregate measurements of neural activity patterns.…”