A dual role for shape skeletons in human vision: perceptual organization and object recognition

Abstract

Shape perception is crucial for object recognition. However, it remains unknown exactly how shape information is represented, and, consequently, used by the visual system. Here, we hypothesized that the visual system incorporates "shape skeletons" to both (1) perceptually organize contours and component parts into a shape percept, and (2) compare shapes to recognize objects. Using functional magnetic resonance imaging (fMRI) and representational similarity analysis (RSA), we found that a model of skeletal similarity explained significant unique variance in the response profiles of V3 and LO, regions known to be involved in perceptual organization and object recognition, respectively. Moreover, the skeletal model remained predictive in these regions even when controlling for other models of visual similarity that approximate low-to high-level visual features (i.e., Gabor-jet, GIST, HMAX, and AlexNet), and across different surface forms, a manipulation that altered object contours while preserving the underlying skeleton. Together, these findings shed light on the functional roles of shape skeletons in human vision, as well as the computational properties of V3 and LO.

Shape coding in occipito-temporal cortex relies on object silhouette, curvature, and medial axis

“…This can be due to a higher spatial inter-subject variability of this representation that has been already observed by Leeds et al (2013), or to a higher collinearity with the low-and high-level models we employed (Figure 1C) that prevents from disentangling its contribution from competing representations. Nonetheless, our result fits previous evidence of medial-axis coding in monkey IT (Hung et al 2012; putative homologue of human LO), human LO (while also controlling for low-level properties: Ayzenberg et al 2019b) and is consistent with our previous MEG study showing that medial-axis processing is limited to a small cluster of right posterior sensors, when controlling for collinearity with low-level and categorical representations (Papale et al 2019).…”

“…Second, a skeletal representation of each stimulus was extracted by performing the medial axis transform (Blum 1973). The spike rate of inferotemporal (IT) neurons in monkey are sensitive to the medial-axis of objects (Hung et al 2012), which also captures behavioral ratings of shape similarity (Ayzenberg et al 2019a;Ayzenberg and Lourenco 2019;Lowet et al 2018) and whose spatiotemporal association with brain activity in humans has been described in several neuroimaging studies (Ayzenberg et al 2019b;Handjaras et al 2017;Leeds et al 2013;Lescroart and Biederman 2013;Papale et al 2019). A third description was obtained by computing the curvature distribution for each object contour.…”

Shape coding in occipito-temporal cortex relies on object silhouette, curvature and medial-axis

“…Nonetheless, our result fits previous evidence of medial-axis coding in monkey IT (Hung et al 2012; putative homologue of human LO), human LO (while also controlling for low-level properties: Ayzenberg et al 2019b) and is consistent with our previous MEG study showing that medial-axis processing is limited to a small cluster of right posterior sensors, when controlling for collinearity with low-level and categorical representations (Papale et al 2019).…”

“…Second, a skeletal representation of each stimulus was extracted by performing the medial axis transform (Blum 1973). The spike rate of inferotemporal (IT) neurons in monkey are sensitive to the medial-axis of objects (Hung et al 2012), which also captures behavioral ratings of shape similarity (Ayzenberg et al 2019a;Ayzenberg and Lourenco 2019;Lowet et al 2018) and whose spatiotemporal association with brain activity in humans has been described in several neuroimaging studies (Ayzenberg et al 2019b;Handjaras et al 2017;Leeds et al 2013;Lescroart and Biederman 2013;Papale et al 2019). A third description was obtained by computing the curvature distribution for each object contour.…”

An fMRI study of visual geometric shapes processing

“…Recently, the brain areas involved in geometric shape processing have been proposed in a distributed network ( Freud et al, 2017 ; Freud and Behrmann, 2020 ) or via a skeletal structure located in V3 and lateral occipital cortex for perceptual organization and object recognition ( Ayzenberg et al, 2019a , b ; Ayzenberg and Lourenco, 2019 ). Bracci and de Beeck (2016) found that shape and category information interacts throughout the ventral and dorsal visual pathways for successful object recognition.…”