Mental rotation and orientation-dependence in shape recognition

Abstract

How do we recognize objects despite differences in their retinal projections when they are seen at different orientations? Marr and Nishihara (1978) proposed that shapes are represented in memory as structural descriptions in objectcentered coordinate systems, so that an object is represented identically regardless of its orientation. An alternative hypothesis is that an object is represented in memory in a single representation corresponding to a canonical orientation, and a mental rotation operation transforms an input shape into that orientation before input and memory are compared. A third possibility is that shapes are stored in a set of representations, each corresponding to a different orientation. In four experiments, subjects studied several objects each at a single orientation, and were given extensive practice at naming them quickly, or at classifying them as normal or mirror-reversed, at several orientations. At first, response times increased with departure from the study orientation, with a slope similar to those obtained in classic mental rotation experiments. This suggests that subjects made both judgments by mentally transforming the orientation of the input shape to the one they had initially studied. With practice, subjects recognized the objects almost equally quickly at all the familiar orientations. At that point they were probed with the same objects appearing at novel orientations. Response times for these probes increased with increasing disparity from the previously trained orientations. This indicates that subjects had stored representations of the shapes at each of the practice orientations and recognized shapes at the new orientations by rotating them to one of the stored orientations. The results are consistent with a hybrid of the second (mental transformation) and third (multiple view) hypotheses of shape recognition: input shapes are transformed to a stored view, either the one at the nearest orientation or one at a canonical orientation. Interestingly, when mirrorimages of trained shapes were presented for naming, subjects took the same time at all orientations. This suggests that mental transformations of orientation can take the shortest path of rotation that will align an input shape and its memorized counterpart, in this case a rotation in depth about an axis in the picture plane.

Priming of Depth-Rotated Objects Depends on Attention and Part Changes

“…Both our and Hayward's (1998) data are not in line with classic view-based accounts which assume a linear increase (e.g., Tarr & Pinker, 1989) or even an accelerated increase (e.g., Poggio & Edelman, 1990) of latencies with increased orientation changes: For attended conditions, a 60∞ rotation in Experiment 2 produced greater priming costs than a 90∞ rotation in Experiment 3 or even a 120∞ rotation in Experiment 1. In addition, to the best of our knowledge, view-based theories would also not predict priming for ignored objects (see Olshausen et al, 1993).…”

“…View-point dependent effects have also been obtained with novel objects rotated in the picture plane (e.g., Tarr & Pinker, 1989 and in depth (e.g., Bulthoff & Edelman, 1992;Hayward & Tarr, 1997;Tarr, 1995;Willems & Wagemans, 2001). In consequence view-based (or image-based) theories (e.g., Bülthoff & Edelman, 1992;Edelman & Intrator, 2003;Poggio & Edelman, 1990;Tarr & Bülthoff, 1995;Tarr & Pinker, 1989) propose that we recognize objects by matching their images to specific holistic views in long-term memory). However, not all object recognition is viewpoint-dependent.…”

Independent representation of parts and the relations between them: Evidence from integrative agnosia.

“…in depth about the vertical axis, consistent with the body of work demonstrating costs to recognition performance associated with changes in viewpoint (Hayward, Wong, & Spehar, 2005;Humphrey & Khan, 1992;Tarr et al, 1997;Tarr & Pinker, 1989). The target-distractor discrimination posed minimal difficulty for the normal participants, regardless of whether the distractors had to be discriminated on the basis of single mismatching parts or on the basis of differences in spatial arrangements of the two parts, and this was true in the icons and no-icons task.…”

Non-holistic coding of objects in lateral occipital complex with and without attention

“…Yet, the specific representational code realizing such invariant representations is still largely unknown. One central question is whether an object is coded as a collection of parts (Hummel and Biederman, 1992;Marr and Nishihara, 1978) or in a viewbased format Poggio and Edelman, 1990;Tarr and Pinker, 1989).…”