Are surface properties integrated into visuohaptic object representations?

Abstract

Object recognition studies have almost exclusively involved vision, focusing on shape rather than surface properties such as color. Visual object representations are thought to integrate shape and color information because changing the color of studied objects impairs their subsequent recognition. However, little is known about integration of surface properties into visuo-haptic multisensory representations. Here, participants studied objects with distinct patterns of surface properties (color in Experiment 1, texture in Experiments 2 & 3) and had to discriminate between object shapes when color/texture schemes were altered in within-modal (visual and haptic) and cross-modal (visual study/haptic test and vice versa) conditions. In Experiment 1, color changes impaired within-modal visual recognition but had no effect on cross-modal recognition, suggesting that the multisensory representation is not influenced by modality-specific surface properties. In Experiment 2, texture changes impaired recognition in all conditions, suggesting that both unisensory and multisensory representations integrate modality-independent surface properties. However, the cross-modal impairment might have reflected either the texture change or a failure to form the multisensory representation. Experiment 3 attempted to distinguish between these possibilities by combining changes in texture with changes in orientation, taking advantage of the known view-independence of the multisensory representation, but the results were not conclusive owing to the overwhelming effect of texture change. The simplest account is that the multisensory representation integrates shape and modality-independent surface properties. However, more work is required to investigate this and the conditions under which multisensory integration of structural and surface properties occurs.

Object and spatial imagery dimensions in visuo-haptic representations

“…The end result was a series of object quadruplets in which shape and texture were exchanged between pairs so that, in each quadruplet, pairs could be used for either shape or texture discrimination. We used difference matrices based on the number of differences in the position and orientation of component blocks to calculate the mean difference in object shape for each of the sets (Lacey et al, 2007, 2009a, 2010). Paired t-tests showed no significant differences between these sets (all p values > .05) and they were therefore considered equally discriminable.…”

“…color and texture, in object representations (Kozhevnikov et al, 2005). We recently showed that texture information is integrated into both visual and haptic representations (Lacey et al, 2010) but did not examine individual differences in this respect. Here, we tested shape discrimination across changes in texture, and texture discrimination across changes in shape, in visual and haptic within-modal conditions.…”

Do things look the way they feel?

“…However, it would be hasty to conclude from this that the visual capacities of the newly sighted are apparent viewpoint-independency of cross-modal object recognition in normal subjects, see Lacey, Peters and Sathian 2007, Lacey et al 2009, and Lacey, Hall and Sathian 2010 14 Held and colleagues do cite this study, but only to challenge an account of their subjects' rapid improvement in the TV task as the result of 'a rapid increase in the visual ability to create a three-dimensional representation, thus allowing for a more accurate mapping between haptic structures and visual ones ' (2011: 552). This may be right, but it overlooks the more fundamental problem that these results raise for the interpretation of their data.…”

Self-supervised learning of depth-based navigation affordances from haptic cues

“…Inspired by Nature, in which visual and haptic sensory feedback are known to be jointly exploited in the Human brain [1] this paper presents a haptic robot-environment interaction system for self-supervised learning of vision skills for safe navigation. For this purpose, the robot is provided with a mechanism to learn a mapping between the volumetric appearance of obstacles, given sensory data provided by a depth sensor, and their bendability, perceived by physically interacting with them with a small antenna (see Fig.…”