Selectivity for food in human ventral visual cortex

Jain, Nidhi; Wang, Aria; Henderson, Margaret; Lin, Ruogu; Prince, Jacob S.; Tarr, Michael J.; Wehbe, Leila

doi:10.1038/s42003-023-04546-2

Cited by 27 publications

(29 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A small number of units appear selective for elements such as sky and window, which can be attributed to processing scenes. Moreover, we also observe single-unit selectivities for the ‘food’ category, for which neural selectivity was also reported in recent works [37, 38, 39], validating our data-driven approach. The non-existence of units aligned with any of the thousands of other semantic concepts beyond these categories in the dissection concept dictionary (e.g.…”

Section: Resultssupporting

confidence: 89%

“…When considering a mask of the ventral visual regions and training a new model to predict the entire region, we recovered the categories detected when considering the ROIs overlapping with this mask: heads (faces), signboards (words) and skin (bodies). We also obtained selectivity for food, which was also reported in recent works [37,38,39]. However, we did not find detectors for the thousands of other semantic concepts present in the large stimulus set and the dissection concept dictionary.…”

Section: A Possible Complete Characterization Of Selectivity In the V...supporting

confidence: 88%

“…Our experiments yielded an a rmative answer to this question, illustrating the success of this unbiased computational approach in revealing the representational structure of the human visual cortex. Further, this experiment also suggests that faces, bodies, text and food [37,38,39] are the categories that selectively activate the highest number of voxels in the ventral visual cortex and provides evidence against the existence of single voxels or regions selective for thousands of other visual concepts covered in our broad network interpretability dictionary beyond these categories.…”

Section: Introductionmentioning

confidence: 75%

See 2 more Smart Citations

Higher visual areas act like domain-general filters with strong selectivity and functional specialization

Khosla

Wehbe

2022

Preprint

View full text Add to dashboard Cite

Investigation of the visual system has mainly relied on a-priori hypotheses to restrict experimental stimuli or models used to analyze experimental data. Hypotheses are an essential part of scientific inquiry, but an exclusively hypothesis-driven approach might lead to confirmation bias towards existing theories and away from novel discoveries not predicted by them. This paper uses a hypothesis-neutral computational approach to study four high-level visual regions of interest (ROIs) selective to faces, places, letters, or body parts. We leverage the unprecedented scale and quality of the Natural Scenes Dataset to constrain neural network models of these ROIs with functional Magnetic Resonance Imaging (fMRI) measurements. We show that using only the stimulus images and the associated activity in an ROI, we are able to train from scratch a neural network that can predict the activity in each voxel of that ROI with an accuracy that beats state-of-the-art models. Moreover, once trained, the ROI-specific networks can reveal what kinds of functional properties emerge spontaneously in their training. Strikingly, despite no category-level supervision, the units in the trained networks act strongly as detectors for semantic concepts like faces or words, thereby providing substantial pieces of evidence for categorical selectivity in these visual areas. Importantly, this selectivity is maintained when training the networks with selective deprivations in the training diet, by excluding images that contain their preferred category. The resulting selectivity in the trained networks strongly suggests that the visual areas do not function as exclusive category detectors but are also sensitive to visual patterns that are typical to their preferred categories, even in the absence of these categories. Finally, we show that our response-optimized networks have distinct functional properties. Together, our findings suggest that response-optimized models combined with model interpretability techniques can serve as a powerful and unifying computational framework for probing the nature of representations and computations in the brain.

show abstract

Section: Resultssupporting

confidence: 89%

Section: A Possible Complete Characterization Of Selectivity In the V...supporting

confidence: 88%

Section: Introductionmentioning

confidence: 75%

See 1 more Smart Citation

Higher visual areas act like domain-general filters with strong selectivity and functional specialization

Khosla

Wehbe

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Over the past several decades, one of the most striking results in visual neuroscience has been the discovery of cortical regions that preferentially encode object categories (e.g., faces, words, scenes, food; Cohen et al (2000), Epstein and Kanwisher (1998), Jain et al (2023), Sergent et al (1992)) and object properties such as object animacy, real-world size, and reachability (Gallivan et al, 2009, Konkle and Caramazza, 2013, Warrington and Shallice, 1984). While these preferences are typically associated with the process of object recognition, visually invariant, functionally-grounded representations of object properties are also critical across a wide variety of real-world behaviors.…”

Section: Introductionmentioning

confidence: 99%

Neural Selectivity for Real-World Object Size In Natural Images

Luo

Wehbe

Tarr

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Real-world size is a functionally important high-level visual property of objects that supports interactions with our physical environment. Critically, real-world-size is robust over changes in visual appearance as projected onto our retinae such that large and small objects are correctly perceived to have different real-world sizes. To better understand the neural basis of this phenomenon, we examined whether the neural coding of real-world size holds for objects embedded in complex natural scene images, as well as whether real-world size effects are present for both inanimate and animate objects, whether low- and mid-level visual features can account for size selectivity, and whether neural size tuning is best described by a linear, logarithmic, or exponential neural coding function. To address these questions, we used a large-scale dataset of fMRI responses to natural images combined with per-voxel regression and contrasts. Importantly, the resultant pattern of size selectivity for objects embedded in natural scenes was aligned with prior results using isolated objects. Extending this finding, we also found that size coding exists for both animate and inanimate objects, that low-level visual features cannot account for neural size preferences, and size tuning functions have different shapes for large versus small preferring voxels. Together, these results indicate that real-world size is an ecologically significant dimension in the larger space of behaviorally-relevant cortical representations that support interactions with the world around us.

show abstract

“…Cortical responses to visual inputs demonstrate organization according to both high-level and low-level stimulus properties. High-level information about images, such as their membership in semantic categories, is reflected in the activation of spatially localized areas of the ventral visual cortex selective for categories such as faces, body parts, places, food, and words ( Downing et al, 2006 ; Epstein & Kanwisher, 1998 ; Jain et al, 2023 ; Kanwisher et al, 1997 ; Khosla et al, 2022 ; McCandliss et al, 2003 ; Pennock et al, 2023 ; Sergent et al, 1992 ). At the same time, low- and mid-level visual features also elicit topographically regular patterns of activation in visual cortex, such as retinotopic maps of spatial position ( Arcaro et al, 2009 ; Sereno et al, 1995 ; Swisher et al, 2007 ) and large-scale maps of selectivity for orientation ( Freeman et al, 2011 ; Issa et al, 2000 ; Sasaki et al, 2006 ), spatial frequency ( Aghajari et al, 2020 ; Bonhoeffer & Grinvald, 1991 ), color ( Conway & Tsao, 2009 ; Zeki, 1973 ), and curvature ( Yue et al, 2014 ; Yue et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

Low-level tuning biases in higher visual cortex reflect the semantic informativeness of visual features

2023

Self Cite

View full text Add to dashboard Cite

Representations of visual and semantic information can overlap in human visual cortex, with the same neural populations exhibiting sensitivity to low-level features (orientation, spatial frequency, retinotopic position) and high-level semantic categories (faces, scenes). It has been hypothesized that this relationship between low-level visual and high-level category neural selectivity reflects natural scene statistics, such that neurons in a given category-selective region are tuned for low-level features or spatial positions that are diagnostic of the region's preferred category. To address the generality of this “natural scene statistics” hypothesis, as well as how well it can account for responses to complex naturalistic images across visual cortex, we performed two complementary analyses. First, across a large set of rich natural scene images, we demonstrated reliable associations between low-level (Gabor) features and high-level semantic categories (faces, buildings, animate/inanimate objects, small/large objects, indoor/outdoor scenes), with these relationships varying spatially across the visual field. Second, we used a large-scale functional MRI dataset (the Natural Scenes Dataset) and a voxelwise forward encoding model to estimate the feature and spatial selectivity of neural populations throughout visual cortex. We found that voxels in category-selective visual regions exhibit systematic biases in their feature and spatial selectivity, which are consistent with their hypothesized roles in category processing. We further showed that these low-level tuning biases are not driven by selectivity for categories themselves. Together, our results are consistent with a framework in which low-level feature selectivity contributes to the computation of high-level semantic category information in the brain.

show abstract

Selectivity for food in human ventral visual cortex

Cited by 27 publications

References 69 publications

Higher visual areas act like domain-general filters with strong selectivity and functional specialization

Higher visual areas act like domain-general filters with strong selectivity and functional specialization

Neural Selectivity for Real-World Object Size In Natural Images

Low-level tuning biases in higher visual cortex reflect the semantic informativeness of visual features

Contact Info

Product

Resources

About