Rotating objects to recognize them: A case study on the role of viewpoint dependency in the recognition of three-dimensional objects

Abstract

Successful object recognition is essential for finding food, identifying kin, and avoiding danger, as well as many other adaptive behaviors. To accomplish this feat, the visual system must reconstruct 3-D interpretations from 2-D "snapshots" falling on the retina. Theories of recognition address this process by focusing on the question of how object representations are encoded with respect to viewpoint. Although empirical evidence has been equivocal on this question, a growing body of surprising results, including those obtained in the experiments presented in this case study, indicates that recognition is often viewpoint dependent. Such findings reveal a prominent role for viewpointdependent mechanisms and provide support for the multiple-views approach, in which objects are encoded as a set of view-specific representations that are matched to percepts using normalization procedures.Just as you could not fully reconstruct a house from photos all taken from a single vantage point, "snapshots" at many angles must be combined to reconstruct a Burgess organism. Conway Morris told me that he managed to reconstruct the curious Wiwaxia-an animal with no modem relatives, and therefore no known prototype to use as a model-by passing countless hours "rotating the damned thing in my mind" from the position of one drawing to the different angle of another, until every specimen could be moved without contradiction from one stance to the next. Then he finally knew that nothing major was missing or out of place.-Stephen J. Gould, Wonderful Life (1989) The human ability to recognize objects is remarkable-under all but the most degraded conditions, we

Image-based object recognition in man, monkey and machine

“…Moreover, the actual evidence for viewpoint-invariance in human visual recognition (as predicted by RBC) is somewhat thin -the most notable experiments that obtain viewpoint invariance for rotations in depth 1 (Biederman and Gerhardstein, 1993) having only limited generalizabilty to other recognition tasks and stimulus sets (Hayward and Tarr, 1995;Tarr and Bülthoff, 1995;Tarr et al, 1997). In contrast, psychophysical and neurophysiological studies from the late 1980s and early 1990s offer a somewhat different conclusion -under a wide variety of experimental conditions, human object recognition performance is strongly viewpoint-dependent across rotations in depth Edelman and Bülthoff, 1992;Humphrey and Khan, 1992;Tarr, 1995). Converging evidence for this result has come from single-cell recording studies in the inferior temporal cortex of monkeys (Logothetis et al, 1995).…”

“…Behavioral results suggest that such models offer only limited explanatory power. For example, several studies Humphrey and Khan, 1992;Tarr, 1995) have demonstrated that, when subjects are trained to recognize novel objects in a small set of viewpoints, not only are the generalization patterns viewpointdependent, but, critically, they are related to the distance between an unfamiliar test view and the nearest familiar view. Such results provide strong evidence for object representations based on multiple image-based views matched to input shapes through normalization processes.…”

“…In part due to this limitation, it is popularly held that both structural-description and image-based models explain elements of human visual recognition. For exam- Tarr, 1995). (b) The bottom pair of objects are qualitatively dissimilar from one another in terms of both image structure and parts (adapted from Hayward, 1998). ple, structural-descriptions providing categorical-level access and image-based mechanisms providing within-class or exemplar-specific level access (Jolicoeur, 1990).…”

“…These conditions attempt to exclude almost all earlier studies, e.g. Bülthoff and Edelman (1992) and Tarr (1995), from consideration as diagnostic of visual recognition. In response to this critique, researchers began to test recognition performance using experimental designs that satisfied Biederman and Gerhardstein's conditions.…”

Non-holistic coding of objects in lateral occipital complex with and without attention

“…Under a viewbased scheme neurons respond most strongly if objects are presented in learned views or configurations. Nevertheless, recognition of objects in varying orientations is thought possible by storing many views of an object Olshausen et al, A C C E P T E D M A N U S C R I P T ACCEPTED MANUSCRIPT 4 1993; Poggio and Edelman, 1990;Tarr and Gauthier, 1998;Tarr, 1995;Ullman, 1998), interpolating across these views (Logothetis et al, 1994;Poggio and Edelman, 1990;Ullman, 1989) or by a distributed neural representation across view-tuned neurons (Perrett et al, 1998).…”