Attention during natural vision warps semantic representation across the human brain

Çukur, Tolga; Nishimoto, Shinji; Huth, Alexander G.; Gallant, Jack L.

doi:10.1038/nn.3381

Cited by 314 publications

(307 citation statements)

References 30 publications

Supporting

Mentioning

286

Contrasting

Unclassified

Order By: Relevance

“…The triplets of words defining the target stimuli did not contain any verbs, in order not to stress the functional differences between animals and tools. Such a 1-back oddball task was orthogonal to the dimensions investigated (size, category, cluster), and this allowed us to disentangle task-dependent processes from the spontaneous mental representations of the words (Cukur et al, 2013). Target stimuli were flashed in the center of the screen three times in a row (each time in a different font among Lucida Fax, Helvetica and Courier, to avoid adaptation): each presentation lasted 0.5 s and the interval between them was 0.2 s for a total of 1.9s for each target stimulus.…”

Section: Testing Proceduresmentioning

confidence: 99%

Word meaning in the ventral visual path: a perceptual to conceptual gradient of semantic coding

et al. 2016

View full text Add to dashboard Cite

valentinaborghesani@gmail.comThe authors declare no competing financial interests. ABSTRACTThe meaning of words referring to concrete items is thought of as a multidimensional representation that includes both perceptual (e.g., average size, prototypical color) and conceptual (e.g., taxonomic class) dimensions. Are these different dimensions coded in different brain regions? In healthy human subjects, we tested the presence of a mapping between the implied real object size (a perceptual dimension) and the taxonomic categories at different levels of specificity (conceptual dimensions) of a series of words, and the patterns of brain activity recorded with functional magnetic resonance imaging in six areas along the ventral occipito-temporal cortical path. Combining multivariate pattern classification and representational similarity analysis, we found that the real object size implied by a word appears to be primarily encoded in early visual regions, while the taxonomic category and subcategorical cluster in more anterior temporal regions. This anteroposterior gradient of information content indicates that different areas along the ventral stream encode complementary dimensions of the semantic space.

show abstract

Section: Testing Proceduresmentioning

confidence: 99%

Word meaning in the ventral visual path: a perceptual to conceptual gradient of semantic coding

et al. 2016

View full text Add to dashboard Cite

show abstract

“…An assumption prevalent in the literature is that comprehension and reading processes are dissociable from sustained attention, and that they can be dissociated by using a control task matched in attentional complexity. Recent research however suggests that attention is a cortex‐wide dynamic process whereby resources are flexibly allocated to the attended function as opposed to a simple mechanism that merely modulates baseline response level [Çukur et al, 2013, Peelen and Kastner, 2014]. But this also implies that subtractive analyses attempting to control for attention factors assuming a static modular nature of attention may be overly conservative.…”

Section: Discussionmentioning

confidence: 99%

Areas activated during naturalistic reading comprehension overlap topological visual, auditory, and somatotomotor maps

Sood

Sereno

2016

Human Brain Mapping

View full text Add to dashboard Cite

Cortical mapping techniques using fMRI have been instrumental in identifying the boundaries of topological (neighbor‐preserving) maps in early sensory areas. The presence of topological maps beyond early sensory areas raises the possibility that they might play a significant role in other cognitive systems, and that topological mapping might help to delineate areas involved in higher cognitive processes. In this study, we combine surface‐based visual, auditory, and somatomotor mapping methods with a naturalistic reading comprehension task in the same group of subjects to provide a qualitative and quantitative assessment of the cortical overlap between sensory‐motor maps in all major sensory modalities, and reading processing regions. Our results suggest that cortical activation during naturalistic reading comprehension overlaps more extensively with topological sensory‐motor maps than has been heretofore appreciated. Reading activation in regions adjacent to occipital lobe and inferior parietal lobe almost completely overlaps visual maps, whereas a significant portion of frontal activation for reading in dorsolateral and ventral prefrontal cortex overlaps both visual and auditory maps. Even classical language regions in superior temporal cortex are partially overlapped by topological visual and auditory maps. By contrast, the main overlap with somatomotor maps is restricted to a small region on the anterior bank of the central sulcus near the border between the face and hand representations of M‐I. Hum Brain Mapp 37:2784–2810, 2016. © 2016 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc.

show abstract

“…Encoding models predict neural or hemodynamic response patterns from computational model parameters 43,44 . This approach has provided insights in domains including visual image identification 30,31,43 , attentional processing 45,46 , and memory 47 . However, encoding models do not support an easy link to behavior 44 .…”

Section: Aligning Complex Behavior Computational Theory and Brain Amentioning

confidence: 99%

Predicting eye movements from deep neural network activity decoded from fMRI responses to natural scenes

Chun

2017

Preprint

View full text Add to dashboard Cite

Computational models of selective spatial attention can reliably predict eye movements to complex images. However, researchers lack a simple way to measure covert representations of spatial attention in the brain and their link to overt eye movement behavior, especially in response to natural scenes. Here, we predict eye movement patterns from spatial priority maps reconstructed from brain activity measured with functional magnetic resonance imaging (fMRI). First, we define a computational spatial attention model using a deep convolutional neural network (CNN) pre-trained for scene categorization. Next, we decode CNN unit activity from fMRI activity and reconstruct spatial priority maps by applying our computational spatial attention model to decoded CNN activity. Finally, we predict eye movements in a subsequent behavioral experiment within and between individuals using reconstructed spatial priority maps. These results demonstrate that features represented in CNN unit activity can guide spatial attention and eye movements, providing a crucial link between CNN models, brain activity, and behavior.

show abstract

Attention during natural vision warps semantic representation across the human brain

Cited by 314 publications

References 30 publications

Word meaning in the ventral visual path: a perceptual to conceptual gradient of semantic coding

Word meaning in the ventral visual path: a perceptual to conceptual gradient of semantic coding

Areas activated during naturalistic reading comprehension overlap topological visual, auditory, and somatotomotor maps

Predicting eye movements from deep neural network activity decoded from fMRI responses to natural scenes

Contact Info

Product

Resources

About