Effects of vision and haptics on categorizing common objects

Abstract

Most research on object recognition and categorization centers on vision. However, these phenomena are likely influenced by the commonly used modality of touch. The present study tested this notion by having participants explore three-dimensional objects using vision and haptics in naming and sorting tasks. Results showed greater difficulty naming (recognizing) and sorting (categorizing) objects haptically. For both conditions, error increased from the concrete attribute of size to the more abstract quality of predation, providing behavioral evidence for shared object representation in vision and haptics.

Visuo-haptic multisensory object recognition, categorization, and representation

“…Similarly, neither vision nor haptics naturally recovered the taxonomic relationships between the natural seashells used by Gaißert and Wallraven (2012): participants distinguished between concrete categories such as whether the shells used were flat or convoluted, rather than between abstract categories such as gastropods (e.g., sea-snail) vs. bivalves (e.g., oyster). If biological relationships were recovered at all, this was mainly contingent on shape similarities, although vision was better than haptics in this respect (Gaißert and Wallraven, 2012) as it was for the abstract categories studied by Haag (2011).…”

“…For example, Haag (2011) used realistically textured models of familiar animals that retained real-life size relations, and required visual and haptic categorization on the basis of size (big/small in real life), domesticity (wild/domestic), and predation (carnivore/herbivore). Errors increased as the basis of categorization moved from concrete (size) to abstract (predation) and were consistently greater in haptics than vision (Haag, 2011).…”

“…For example, Haag (2011) used realistically textured models of familiar animals that retained real-life size relations, and required visual and haptic categorization on the basis of size (big/small in real life), domesticity (wild/domestic), and predation (carnivore/herbivore). Errors increased as the basis of categorization moved from concrete (size) to abstract (predation) and were consistently greater in haptics than vision (Haag, 2011). Similarly, neither vision nor haptics naturally recovered the taxonomic relationships between the natural seashells used by Gaißert and Wallraven (2012): participants distinguished between concrete categories such as whether the shells used were flat or convoluted, rather than between abstract categories such as gastropods (e.g., sea-snail) vs. bivalves (e.g., oyster).…”

Visual and Tactile Sensory Systems Share Common Features in Object Recognition

“…While much is known about visual shape processing ( Haushofer et al, 2008 ; Peelen et al, 2014 ), less information is available regarding tactile shape processing ( Klatzky et al, 1985 ; Hernández-Pérez et al, 2017 ; Metzger et al, 2019 ). A series of studies comparing visual and tactile perceptual spaces with familiar objects have revealed that the human perception of familiar objects is not solely determined by the physical features of objects but is influenced by high-level cognitive abilities, including memory ( Amedi et al, 2002 ; Norman et al, 2008 ; Haag, 2011 ; Metzger and Drewing, 2019 ) and prior knowledge of objects for integrating sensory systems ( Ernst and Bülthoff, 2004 ). While other studies have used parametric shape models, such as shell-shaped 3D objects ( Gaißert et al, 2008 , 2010a , 2011 ; Gaißert and Wallraven, 2012 ), it is difficult to capture the shape complexity of natural objects with parametric approaches and avoid possible confounds or special cases in object shapes ( Haushofer et al, 2008 ; Lee Masson et al, 2016 ).…”

Haptic Categorical Perception of Shape

“…Lederman & Klatzky [26] analyzed how humans categorize common, everyday objects, while Haag [27] compared visual and haptic categorization of toy objects, which resembled miniaturized animals. While these studies have shown that humans are impressively good at haptic recognition and classification of familiar objects, such stimulus sets have a number of limitations.…”