Seeing structure: Shape skeletons modulate perceived similarity

Abstract

An intrinsic part of seeing objects is seeing how similar or different they are relative to one another. This experience requires that objects be mentally represented in a common format over which such comparisons can be carried out. What is that representational format? Objects could be compared in terms of their superficial features (e.g., degree of pixel-by-pixel overlap), but a more intriguing possibility is that they are compared on the basis of a deeper structure. One especially promising candidate that has enjoyed success in the computer vision literature is the shape skeleton-a geometric transformation that represents objects according to their inferred underlying organization. Despite several hints that shape skeletons are computed in human vision, it remains unclear how much they actually matter for subsequent performance. Here, we explore the possibility that shape skeletons help mediate the ability to extract visual similarity. Observers completed a same/different task in which two shapes could vary either in their skeletal structure (without changing superficial features such as size, orientation, and internal angular separation) or in large surface-level ways (without changing overall skeletal organization). Discrimination was better for skeletally dissimilar shapes: observers had difficulty appreciating even surprisingly large differences when those differences did not reorganize the underlying skeletons. This pattern also generalized beyond line drawings to 3-D volumes whose skeletons were less readily inferable from the shapes' visible contours. These results show how shape skeletons may influence the perception of similarity-and more generally, how they have important consequences for downstream visual processing.

Getting “fumpered”: Classifying objects by what has been done to them

“…To test more specifically for the role of mid-level texture-like statistical features in the perception of generative processes, we created a new set of twodimensional objects to pit them against Euclidean shape similarity-a prime source of object classification (e.g., Lowet, Firestone, & Scholl, 2018;Wilder, Feldman, & Singh, 2011). As a control, after asking participants to classify the stimuli, we used apparent motion to measure low-level perceptual shape similarity between stimuli.…”

A dual role for shape skeletons in human vision: perceptual organization and object recognition

“…Similarly, behavioral research with humans has shown that participants extract the skeleton of 2D shapes (Firestone & Scholl, 2015;Kovács, Fehér, & Julesz, 1998;Psotka, 1978), even in the presence of border perturbations and illusory contours (Ayzenberg, Chen, Yousif, & Lourenco, 2019). Other research has shown that skeletal models are predictive of human object recognition (Destler, Singh, & Feldman, 2019;Lowet, Firestone, & Scholl, 2018;Wilder, Feldman, & Singh, 2011), even when controlling for other models of vision . Although these studies provide evidence that skeletons are associated with perceptual organization and object recognition, at least when these functions are tested independently, it remains unknown whether shape skeletons are involved in both functions.…”

Skeletal descriptions of shape provide unique perceptual information for object recognition

“…Skeletal similarity between every object was calculated in 3D, object-centered, space as the mean Euclidean distance between each point on one skeleton and the closest point on the second skeleton following maximal alignment (see Methods). We chose to test a 3D skeletal description because of behavioral 48 and neural 49 evidence for 3D object-centered representations in the visual system, which include a sensitivity to 3D skeletal structures 36,37 .…”