Mid-level visual features underlie the high-level categorical organization of the ventral stream

Abstract

Human object-selective cortex shows a large-scale organization characterized by the high-level properties of both animacy and object size. To what extent are these neural responses explained by primitive perceptual features that distinguish animals from objects and big objects from small objects? To address this question, we used a texture synthesis algorithm to create a class of stimuli-texforms-which preserve some mid-level texture and form information from objects while rendering them unrecognizable. We found that unrecognizable texforms were sufficient to elicit the large-scale organizations of object-selective cortex along the entire ventral pathway. Further, the structure in the neural patterns elicited by texforms was well predicted by curvature features and by intermediate layers of a deep convolutional neural network, supporting the mid-level nature of the representations. These results provide clear evidence that a substantial portion of ventral stream organization can be accounted for by coarse texture and form information without requiring explicit recognition of intact objects.

Selectivity for mid‐level properties of faces and places in the fusiform face area and parahippocampal place area

“…It is possible that these mid‐level properties also contribute to the patterns of response in category‐selective regions. Recently, images that preserve mid‐level properties of objects, but were not recognizable, elicited similar patterns of neural response to animacy and real‐world size found for intact objects (Long et al.,). When evaluating these possibilities, it is important to recognize that high‐level, mid‐level and low‐level contributions to the observed representational structure are not mutually exclusive.…”

“…Adaptation (different-same) changed very little across categories or image types. Colours reflect different subjects, with the group mean shown in black to animacy and real-world size found for intact objects (Long et al, 2018). When evaluating these possibilities, it is important to recognize that high-level, mid-level and low-level contributions to the observed representational structure are not mutually exclusive.…”

“…Despite the fact that images were unrecognizable, locally‐scrambled images, but not globally‐scrambled images, were found to elicit similar category‐selective patterns of response across the ventral visual pathway (Coggan, Baker, & Andrews, ; Coggan, Liu et al., ). More recently, Long, Yu, and Konkle () showed that images that preserve mid‐level properties of objects, but were not recognizable elicited patterns of neural response to variation in animacy and real‐world size that were comparable to intact objects.…”

A data‐driven approach to stimulus selection reveals an image‐based representation of objects in high‐level visual areas

“…Moreover, the pattern of response to each metameric object cluster was similar to the pattern of response elicited by the corresponding category-defined, object cluster. Given this, it is possible that other models that incorporate mid-level representations of objects may predict patterns of neural response more accurately than GIST (Guclu & van Gerven, 2015;Khaligh-Razavi & Kriegeskorte, 2014;Leeds, Seibert, Pyles, & Tarr, 2013;Long et al, 2018;Yamins et al, 2014). We found that the ability to find objects that had similar image properties to a category, but were not a member of that category varied for different categories.…”

“…Electrophysiological studies in nonhuman primates also suggest that the topography of higher visual areas is based on a continuous mapping of image features rather than discrete object representations (Tanaka, 1996). For example, categoryselective patterns of response are still evident when images have been scrambled in a way that preserves some of their visual properties, but removes their semantic properties (Andrews et al, 2010;Coggan, Liu, Baker, & Andrews, 2016;Long, Yu, & Konkle, 2018;Watson, Andrews, & Hartley, 2017). Neuroimaging studies have also shown that differences in the visual properties of objects can explain a significant amount of the variance in high-level regions of visual cortex (Coggan et al, 2019;Levy et al, 2001;Rice et al, 2014;Nasr et al, 2014;Sormaz et al, 2016).…”