Structure from motion: A tolerance analysis

Abstract

For a proper evaluation of the performance of the visual system one needs to analyze the stimuli. From such an analysis it should become clear whether the information required for a certain task is available and to what extent. If we do not perform such an analysis we cannot decide whether the limits reached by the visual system are set by stimulus limitations or by limits of the visual system. Koenderink and van Doorn (1987) performed a tolerance analysis to measure the extent to which certain information is available in velocity flow fields. Here we will do the same for an important class of structure-from-motion stimuli, using multiple frames. As an example the tolerance analysis will be applied to two widely known experiments. In this introduction we will describe the structurefrom-motion problem and the role a tolerance analysis can play in the investigation of it.A tolerance analysis of a system consists of an analysis of the degree to which the output of the system is resistant against perturbations in the input. When the output fluctuates a lot with small fluctuations in the input, it can be said that the output is not well defined. To develop a model of the visual system one can make use of an ideal detector model. By ideal detector we mean a system that reaches the optimal performance with respect to some criterion. Note that what is an ideal detector heavily depends on the definition. By comparing the performance of the visual system with an ideal detector model one can determine the extent to which the visual system approaches ideal performance.In this article we will show the use of a tolerance analysis in structure-from-motion (SfM) experiments. In SfM the input to the visual system consists of projections of the 3-D world on the retina (we discard input from other sensory systems). The fact that 3-D properties have to be inferred from projections highly limits performance in 3-D tasks. We will consider a world consisting of a number of landmarks. To obtain the 3-D structure of a number of landmarks from its projections, assumptions have to be made about the 3-D structure or the 3-D movement between the projections. Like many others, we assume that the 3-D movement is a rigid one. We assume that the projected locations of the landmarks will be available up to limited precision. More precisely, we assume that the locations available to the system are drawn from normal distributions around the actual ones. No other assumptions about the 3-D structure or the movement are being used. We will explain a method to develop an ideal detector model for rigidly moving objects given these assumptions. We will develop such a model for a situation in which the object rotates rigidly about an axis parallel to the image plane. By comparing performance of the visual system with that of the ideal detector model we can find out the degree to which it resembles the ideal detector model. Taking 2-D measures, which are more directly related to the stimulus, has several advantages over taking 3-D measures of performance ...

The visual perception of rigid motion from constant flow fields

“…In order to assess the actual extent of3-D structural deformation depicted in these constant flow field displays, it is useful to employ a tolerance analysis that has recently been developed by Hogervorst, Kappers, and Koenderink (1996) to test the rigidity of an object's projected motion after it has been transformed to a pattern of parallel trajectories by the removal ofimage rotation. The basic idea is quite simple and powerful.…”

Monocular discrimination of rigidly and nonrigidly moving objects

“…Still, using the planarity constraint is often sufficient to discriminate rigid from nonrigid transformations. More details about this method can be found in Hogervorst et al (1996). ,r n ) and (l/>l' ... , l/>n).…”

“…In some ofthe experiments of Norman and Todd (1993), subjects perceived a rigidly moving object even though they were looking at the projections of a nonrigidly moving object. We (Hogervorst, Kappers, & Koenderink, 1996) have shown that in those cases, the projections were very similar to those of a rigidly moving object. The visual system is apparently biased toward perceiving rigidly moving objects.…”

“…Ullman (1979) has shown that three parallel projections ofat least four points uniquely define the 3-D structure up to a reflection about the image plane (and a uniform scaling). However, whether two perspective projections or three parallel projections contain the information that the observer needs to reach a certain level ofperformance in a 3-D task is a matter of tolerances (Hogervorst et al, 1996;Koenderink & van Doorn, 1987).…”

Discerning nonrigid 3D shapes from motion cues

“…Humans seem to be quite good at disentangling shape changes from object motion and characterizing motion as tumbling, rolling, swaying, stretching, leaping, spinning, flapping, dancing, kicking, bucking, jerking, sliding, gliding, tripping, or shaking. A large number of studies have examined human perception of rigid 3D shapes from motion cues (1)(2)(3)(4)(5); however, very few have examined nonrigid shape perception (6)(7)(8), and these have not dealt with what shapes are perceived.…”