Computational models of category-selective brain regions enable high-throughput tests of selectivity

Abstract

Cortical regions apparently selective to faces, places, and bodies have provided important evidence for domain-specific theories of human cognition, development, and evolution. But claims of category selectivity are not quantitatively precise and remain vulnerable to empirical refutation. Here we develop artificial neural network-based encoding models that accurately predict the response to novel images in the fusiform face area, parahippocampal place area, and extrastriate body area, outperforming descriptive models and experts. We use these models to subject claims of category selectivity to strong tests, by screening for and synthesizing images predicted to produce high responses. We find that these high-response-predicted images are all unambiguous members of the hypothesized preferred category for each region. These results provide accurate, image-computable encoding models of each category-selective region, strengthen evidence for domain specificity in the brain, and point the way for future research characterizing the functional organization of the brain with unprecedented computational precision.

Emergent selectivity for scenes, object properties, and contour statistics in feedforward models of scene-preferring cortex

“…We used convolutional neural networks (CNNs) and fMRI data to create image-computable voxelwise encoding models of mid-level feature tuning in the PPA. CNNs are theoretical models of the core information-processing mechanisms implemented by biological neural populations, and they are the leading computational models of human visual cortex 28 , including scene-selective areas 24, 25, 29 . They perform a set of biologically plausible mathematical operations, and their hierarchical, convolutional architecture is inspired by the primate visual system.…”

“…Another key finding showed that the category-selective organization of object representations in the macaque ventral stream can be mapped onto the first two principal components of a feedforward CNN and may thus naturally arise from the statistical structure of visual feature representations 36 . It has also recently been shown the feedforward neural network models provide more accurate predictions of response preferences in category-selective visual regions than descriptive models of image categories or the predictions of experts in the field 25 . More broadly, multiple studies have found that the representations of the human ventral stream are better explained by perceptual features than by the abstract properties that underlie category identity or human intuitions about semantic similarity 25, 29, 43–45 .…”

“…A powerful approach for addressing such questions is the use of high-throughput experiments on neural network models of the visual system 24–27 . It has previously been shown that image-computable neural networks can yield remarkably accurate encoding models of category-selective visual cortex, including the PPA 24, 25 , but no studies have directly examined whether selectivity for both high-level scene and object properties and low-level contour statistics can emerge from a unified representational model. To address this question, we fit feedforward neural network models to the scene-evoked fMRI responses of the PPA.…”

Beyond linear regression: mapping models in cognitive neuroscience should align with research goals

“…Finally, some researchers are trying to build accurate models of the brain activity that, in essence, enable simulations of neuroscience experiments (e.g., Khosla & Wehbe, 2021;Ratan Murty et al, 2021). This type of modeling is especially important in cases when experimental data are expensive or hard to obtain: with a high-accuracy model of brain responses, a researcher can run thousands of experiments in silico, refine their hypothesis, and then test the final prediction in vivo.…”

“…However, nameability of features may be an overly restrictive metric as it limits our understanding to a vocabulary that is heavily biased by a priori hypotheses and may not include words for the concepts we actually need (Buzsáki, 2019). For instance, recent work has shown that (a) neurons typically described as "face-responsive" respond more strongly to artificial images produced by DNNs than to natural images described by the word "face" (Ponce et al, 2019) and (b) a neural-network-based linearized model of activity in the fusiform face area predicts responses to faces better than label-based models (Ratan Murty et al, 2021), suggesting that simple verbal features cannot provide a full account of neural activity. To overcome the limitation of using individual nameable features, many researchers have instead started to use high-dimensional feature sets.…”