Representation and recognition of the spatial organization of three-dimensional shapes

Abstract

The human visual process can be studied by examining the computational problems associated with deriving useful information from retinal images. In this paper, we apply this approach to the problem of representing three-dimensional shapes for the purpose of recognition. 1. Three criteria, accessibility, scope and uniqueness , and stability and sensitivity , are presented for judging the usefulness of a representation for shape recognition. 2. Three aspects of a representation’s design are considered, (i) the representation’s coordinate system, (ii) its primitives, which are the primary units of shape information used in the representation, and (iii) the organization the representation imposes on the information in its descriptions. 3. In terms of these design issues and the criteria presented, a shape representation for recognition should: (i) use an object-centred coordinate system, (ii) include volumetric primitives of varied sizes, and (iii) have a modular organization. A representation based on a shape’s natural axes (for example the axes identified by a stick figure) follows directly from these choices. 4. The basic process for deriving a shape description in this representation must involve: (i) a means for identifying the natural axes of a shape in its image and (ii) a mechanism for transforming viewer-centred axis specifications to specifications in an object-centred coordinate system. 5. Shape recognition involves: (i) a collection of stored shape descriptions, and (ii) various indexes into the collection that allow a newly derived description to be associated with an appropriate stored description. The most important of these indexes allows shape recognition to proceed conservatively from the general to the specific based on the specificity of the information available from the image. 6. New constraints supplied by a conservative recognition process can be used to extract more information from the image. A relaxation process for carrying out this constraint analysis is described.

Independent representation of parts and the relations between them: Evidence from integrative agnosia.

“…Several theories have argued that the parts of many objects can be modeled as volumetric components, such as generalized cylinders (Biederman, 1987;Binford, 1971;Marr & Nishihara, 1978). The parts used to construct the objects in the present study are roughly akin to geons, which are simple threedimensional volumetric shapes made of local features, and it is from these geons that object representations are built (Biederman, 1987).…”

“…One theoretical approach posits that a threedimensional object is represented explicitly by a small number of volumetric primitives combined with their spatial relationships (Biederman, 1987;Hoffman & Richards, 1984;Hummel, 1994;Marr & Nishihara, 1978;Palmer, 1977). Although earlier versions of this structural description approach were challenged on empirical, theoretical, and computational grounds (Edelman, 1997), more recent versions address some of the criticisms by allowing for both preserved metric information about size and shape, which permits more robust within-category discrimination (Biederman, Subramaniam, Bar, Kalocsai, & Fiser, 1999), and some consideration of the role of experience in object perception (Hummel & Stankiewicz, 1996, which permits differential performance on different classes of objects as a function of expertise (Palmeri & Gauthier, 2004).…”

Priming of Depth-Rotated Objects Depends on Attention and Part Changes

“…Viewpoint invariant recognition is usually accounted for by theories that propose part-based representations (e.g., Biederman, 1987;Hummel & Biederman, 1992;Marr & Nishihara, 1978;Palmer, 1977). According to this approach the visual system represents objects as structural descriptions, which specify an object's parts, such as generalized cylinders (Marr & Nishihara, 1978) or geons (Biederman, 1987), in terms of their spatial relations to one another.…”

“…According to this approach the visual system represents objects as structural descriptions, which specify an object's parts, such as generalized cylinders (Marr & Nishihara, 1978) or geons (Biederman, 1987), in terms of their spatial relations to one another. For example, a structural description would represent the shape of a coffee mug as a "curved cylinder" (i. e., the handle) "side-attached" to a "vertical straight cylinder" (i. e., the body, see Biederman, 1987).…”

Integration of depth modules: Stereopsis and texture

“…These cues have been investigated extensively in isolation (Kaufman, 1974;Marr, 1982) under the assumption that depth is processed in separate modules corresponding to the different sources of three-dimensional information. The "2iD sketch" (Marr & Nishihara, 1978) is one explicit proposal for the form of representation that could result from the combination of independent depth modules. It consists of a map of the distance and orientation of surface points with respect to the viewpoint.…”