3D model acquisition from extended image sequences

Abstract

Abstract. A method for matching image primitives through a sequence is described, for the purpose of acquiring 3D geometric models. The method includes a novel robust estimator of the trifocal tensor, based on a minimum number of token correspondences across an image triplet; and a novel tracking algorithm in which corners and line segments are matched over image triplets in an integrated framework. The matching techniques are both robust (detecting and discarding mismatches) and fully automatic. The matched tokens are used to compute 3D structure, which is initialised as it appears and then recursively updated over time. The approach is uncalibrated -camera internal parameters and camera motion are not known or required. Experimental results are provided for a variety of scenes, including outdoor scenes taken with a hand-held camcorder. Quantitative statistics are included to assess the matching performance, and renderings of the 3D structure enable a qualitative assessment of the results. 1Introduction The aim of this work is to recover 3D models from long uncalibrated monocular image sequences. These models will be used for graphics and virtual reality applications. The sequences are generated by circumnavigating the object of interest (e.g. a house) acquiring images with a camcorder. Neither the internal calibration of the camera nor the motion of the camera are known. In particular the motion is unlikely to be smooth. The focus of this paper is on the matching and tracking of image primitives which underpins the structure recovery.We build on the work of a number of previous successful systems which have recovered structure and motion from tracked image primitives (tokens). Coarsely these systems can be divided into those that use sequential [1,3,4,10,19,28], and those that use batch updates [13,15,23]. Here the matching and structure update are sequential. However, the basic unit is an image triplet, rather than the more usual image pair. It is in this area of tracking technology [26,27] that we have made the most significant innovations. A finite displacement (several cms) between views prohibits the utilisation of simple nearest neighbour token matching strategy between consecutive images. In this work:1. Corner and line segments are matched simultaneously over image triplets in an integrated framework, by employing the trifocal tensor [14,21,22]. 2. A robust estimator for the trifocal tensor is developed, with the tensor instantiated over three views using a minimal set (six) of point matches and a RANSAC scheme.

Coarse to over-fine optical flow estimation

“…Optical flow calculation for range images, also known as range flow (Spies et al, 2002), is expected to have the same benefit as optical flow from this approach. Depth from stereo (Beardsley et al, 1996) relies on extracting information from multiple images, and requires piecewise smoothness at object boundaries. This similarity to optical flow makes it likely to benefit from this approach as well.…”

Piecewise-consistent color mappings of images acquired under various conditions

“…Co-segmentation of planes was addressed by several studies, where all of them, including ours, compute the homography as a first step using corresponding features. In previous studies the plane's segments are detected by assuming identical colors [16], Delaunay triangulation [1], or by setting the boundaries using the intersection line between planes as determined by their homographies [18]. The first approach makes a strong assumption about scene illumination (identical colors) while the other approaches make a strong assumption about scene structure (e.g., hole-free planes or intersecting planes).…”

Untitled

“…This makes the data association problem harder: most existing approaches rely on the temporal continuity of an image stream to track features over time (Deriche & Faugeras, 1990;Tomasi & Kanade, 1992;Zhang & Faugeras, 1992;Cox, 1993), or otherwise constrain the data association problem (Beardsley, Torr, & Zisserman, 1996). 4.…”

“…Finally, methods based on the robust recovery of epipolar geometry, e.g. using RANSAC (Beardsley, Torr, & Zisserman, 1996;Torr, Fitzgibbon, & Zisserman, 1998) can cope with larger inter-frame displacements and can be very effective in practice.…”