Reconstruction of visual appearance

Abstract

This paper presents a method for reconstructing the visual appearance of a scene from any viewpoint, given a collection of stereo pairs. A representation is suggested which is flexible and efficient, and algorithms to recover three dimensional structure and grey-level detail by combining a number of two-dimensional views are discussed. Some results using real and simulated data are given. This paper presents a method for reconstructing the visual appearance of a scene from any viewpoint given a collection of stereo pairs. For example, figure 1 shows two stereo pairs of the same building. The objective is to generate a new view from this data as in figure 2 (see below). The method has a number of properties:• Occlusion is dealt with explicitly and all surface fragments visible from at least one stereo pair are correctly rendered in the output.• Surface shape and texture are obtained from the best available data.• The stereo pairs are pre-compiled into a concise representation from which reconstruction is an 0(n) process (n = number of stereo pairs).Reconstruction of visual appearance has been used extensively for visualisation of terrain models [1]. Some authors have also used reconstruction to show the result of stereo algorithms [2]. The Mosaic system [3] used stereo and geometric reasoning to generate wire-frame models of urban scenes. Although domain specific to aerial photographs of buildings, some results using grey-level rendering were achieved. Feature tracking [4,5,6,7] has been used to recover three-dimensional structure on a frame-by-frame basis. model generation of continuous objects, with accurate depth information using a controlled light source. SYSTEM OVERVIEWThe system works in two parts. Steps 1, 2, 3 and 4 are performed off-line from the available stereo pairs.Step 5 is performed for each reconstruction required.1. Production of a depth map from each stereo pair. The relative camera orientation is determined and corresponding points are found for each pair. These are projected by triangulation to give a grid of threedimensional locations.2. Computation of relative positions of stereo pairs. Overlapping regions of the surfaces generated from the stereo pairs allow the relative positions of their viewpoints to be computed.3. Detection of depth map discontinuities. By comparing depth maps, discontinuities in the interpolated depth maps are located and marked to avoid unwanted interpolation during reconstruction. The result is a set of viewpoint-dependent depth maps with discontinuities, somewhat similar to Marr's concept [12] of a 2|d sketch.

Tracking Curved Objects by Perspective Inversion

“…• Grey-level modelling Methods for generating these models automatically from real images are being studied by North [7,8]. The use of non-uniform sampling (i.e.…”

“…It is suggested that such grey-level models can be computed automatically (e.g. [7,8]) and rendered in real time. The expense of this is partly offset by savings when inverting the perspective transform (since more consistent error vectors yield faster iterative solutions).…”

Optimal Surface Fusion

“…This occurs when the measured surface occludes itself on projection, or when some surface already represented in the model is occluding. The self-occlusion problem can be solved by z-buffering during projection [5] but occlusion by other surfaces is more problematic.…”

“…The problem of estimating a single optimal surface from noisy measurements occurs in many vision and robotics applications [1,2,3,4]. Here it is considered in the context of building a description of a complex object or environment using stereo reconstruction from many viewpoints [5]. A definition is offered of the optimal surface to represent a set of measured surfaces, and the paper shows how it may be found using the Kalman filter framework.…”