This paper presents a method for reconstructing the visual appearance of a scene from any viewpoint, given a collection of stereo pairs. A representation is suggested which is flexible and efficient, and algorithms to recover three dimensional structure and grey-level detail by combining a number of two-dimensional views are discussed. Some results using real and simulated data are given. This paper presents a method for reconstructing the visual appearance of a scene from any viewpoint given a collection of stereo pairs. For example, figure 1 shows two stereo pairs of the same building. The objective is to generate a new view from this data as in figure 2 (see below). The method has a number of properties:• Occlusion is dealt with explicitly and all surface fragments visible from at least one stereo pair are correctly rendered in the output.• Surface shape and texture are obtained from the best available data.• The stereo pairs are pre-compiled into a concise representation from which reconstruction is an 0(n) process (n = number of stereo pairs).Reconstruction of visual appearance has been used extensively for visualisation of terrain models [1]. Some authors have also used reconstruction to show the result of stereo algorithms [2]. The Mosaic system [3] used stereo and geometric reasoning to generate wire-frame models of urban scenes. Although domain specific to aerial photographs of buildings, some results using grey-level rendering were achieved. Feature tracking [4,5,6,7] has been used to recover three-dimensional structure on a frame-by-frame basis. model generation of continuous objects, with accurate depth information using a controlled light source.
SYSTEM OVERVIEWThe system works in two parts. Steps 1, 2, 3 and 4 are performed off-line from the available stereo pairs.Step 5 is performed for each reconstruction required.1. Production of a depth map from each stereo pair. The relative camera orientation is determined and corresponding points are found for each pair. These are projected by triangulation to give a grid of threedimensional locations.2. Computation of relative positions of stereo pairs. Overlapping regions of the surfaces generated from the stereo pairs allow the relative positions of their viewpoints to be computed.3. Detection of depth map discontinuities. By comparing depth maps, discontinuities in the interpolated depth maps are located and marked to avoid unwanted interpolation during reconstruction. The result is a set of viewpoint-dependent depth maps with discontinuities, somewhat similar to Marr's concept [12] of a 2|d sketch.