Visual and Tactile Sensory Systems Share Common Features in Object Recognition

Abstract

Although we use our visual and tactile sensory systems interchangeably for object recognition on a daily basis, little is known about the mechanism underlying this ability. This study examined how 3D shape features of objects form two congruent and interchangeable visual and tactile perceptual spaces in healthy male and female participants. Since active exploration plays an important role in shape processing, a virtual reality environment was used to visually explore 3D objects called digital embryos without using the tactile sense. In addition, during the tactile procedure, blindfolded participants actively palpated a 3D-printed version of the same objects with both hands. We first demonstrated that the visual and tactile perceptual spaces were highly similar. We then extracted a series of 3D shape features to investigate how visual and tactile exploration can lead to the correct identification of the relationships between objects. The results indicate that both modalities share the same shape features to form highly similar veridical spaces. This finding suggests that visual and tactile systems might apply similar cognitive processes to sensory inputs that enable humans to rely merely on one modality in the absence of another to recognize surrounding objects.

Evidence for Independent Processing of Shape by Vision and Touch

“…These results may seem to contradict a recent study by Tabrik et al (2021) . In that study, a great deal of similarity was found between the chosen shape metrics and the self-reported measures of perceptual similarity in both within-modal visual and within-modal haptic comparisons (different groups of participants were used for these tasks, so no cross-modal comparisons were possible).…”

“…For example, it was possible (though exceedingly unlikely) to have the same shape presented five times in a row. Consistent with previous experiments comparing visual and haptic stimuli ( Newell et al, 2001 ; Lacey et al, 2007 , 2009 ; Tabrik et al, 2021 ), participants were given double the time to explore haptic (6 s) as visual (3 s), after which point the stimuli were removed. They could report their decision at any time during or after the stimulus presentation.…”

Evidence for Independent Processing of Shape by Vision and Touch

“…These results may seem to contradict a recent study by Tabrik and colleagues (2021). In that study, a great deal of similarity was found between the chosen shape metrics and the self-reported measures of perceptual similarity in both within-modal visual and within-modal haptic comparisons (different groups of participants were used for these tasks, so no cross-modal comparisons were possible).…”

“…Previous work (Deshpande et al, 2010; Lacey et al, 2010) has indicated that the networks involved in haptic object recognition are similar to visual object recognition only when the shapes are familiar. This may also explain the results found in Tabrik et al (2021) where shapes were likely more familiar and greater similarity was found between visual and haptic processing as compared to the present study. Further work will seek to determine the extent to which stimulus familiarity impacts which metrics best predict human behavior.…”

“…For example, it was possible (though exceedingly unlikely) to have the same shape presented 5 times in a row. Consistent with previous experiments comparing visual and haptic stimuli (Newell et al, 2001;Lacey et al, 2007Lacey et al, , 2009Tabrik et al, 2021), participants were given double the time to explore haptic (6 s) as visual (3 s), after which point the stimuli were removed. They could report their decision at any time during or after the stimulus presentation.…”