Viewpoint Dependence in Visual and Haptic Object Recognition

Abstract

Abstract-On the whole, people recognize objects best when they see the objects from a familiar view and worse when they see the objects from views that were previously occluded from sight. Unexpectedly, we found haptic object recognition to be viewpoint-specific as well, even though hand movements were unrestricted. This viewpoint dependence was due to the hands preferring the back "view" of the objects. Furthermore, when the sensory modalities (visual vs. haptic) differed between learning an object and recognizing it, recognition performance was best when the objects were rotated back-to-front between learning and recognition. Our data indicate that the visual system recognizes the front view of objects best, whereas the hand recognizes objects best from the back.People explore and navigate through their environment mainly using sight and touch. In order to guide actions and interactions with objects, information acquired from the visual and the haptic systems must converge to form a coherent percept. What might be the nature of the representations underlying each sensory system, in order to allow this convergence? If the visual and haptic representations of an object are qualitatively different, a translation process must be involved for the two systems to communicate. The presence of a translator implies that moving information between the visual and haptic systems can be inefficient. In the experiments reported in this article, we studied the nature of object representation in each sensory system and the interaction between these systems. Specifically, we considered whether representations in each system are either dependent on or invariant to viewpoint.

Differences between Early-Blind, Late-Blind, and Blindfolded-Sighted People in Haptic Spatial-Configuration Learning and Resulting Memory Traces

“…We normally look at what our hands are doing, and attentional resources could become dominated by the visual channel in these circumstances. In turn, beneficial effects of visual experience on haptic identification of particularly (familiar) 3-D stimuli (Shimizu et al 1993) may result from similar (Newell et al 2001;James et al 2002) or common representations of haptic and visual form (Reales and Ballesteros 1999) or from a unidirectional enhancement, ie visual experience enhances haptic processing (cf Sathian et al 1997;Amedi et al 2001;Pascual-Leone and Hamilton 2001). In addition, for the inferior performance of the early-blind in 2-D identification tasks (Heller 1989b, experiment 2;Shimizu et al 1993), several theoretical explanations have been put forward, such as problems with making decisions (Heller 1989b), and/or dependence on egocentric coding (Millar 1981;Gaunet et al 1997).…”

Multisensory recognition of actively explored objects.

“…(Replotted from Newell et al 2001. ) In an effort to examine what shape information is encoded in vision and touch, respectively, in a recent recognition memory study we used unfamiliar Lego™ objects as stimuli presented in a fixed position (Newell et al, 2001) (c.f. Figure 2).…”

“…For the encoding of shape information, haptics is generally slower than vision (e.g., Jones, 1981;Newell et al, 2001;Woods, O'Modhrain, & Newell, 2004) and consequently performance was worse for haptic than for visual learning. In our first Experiment (1a), the learning times for both vision and touch were limited to 30 s each.…”

“…We used the same stimulus set of unfamiliar objects as described in Newell et al (2001): All objects were composed of six, same-sized Lego bricks (the 8-bit brick measuring 3 cm x 1.5 cm each) stacked on top of each other, so that long and short faces were alternating. Each object was defined by a unique configuration of these bricks and had a middle vertical column of 4 bits (see Fig.…”

The Influence of Viewpoint and Object Detail in Blind People When Matching Pictures to Complex Objects

“…Thus, Newell et al (2002) reported that the backs of objects were the preferred vantage points for touch. Viewpoint is closely linked to linear perspective, since perspective will vary with vantage point (Heller et al 2002.…”