Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares

Abstract

The Cambridge laboratory became operational in 1988 and is located at One Kendall Square, near MIT. CRL engages in computing research to extend the state of the computing art in areas likely to be important to Digital and its customers in future years. CRL's main focus is applications technology; that is, the creation of knowledge and tools useful for the preparation of important classes of applications. CRL AbstractThe simultaneous recovery of 3D shape and motion from image sequences is one of the more difficult problems in computer vision. Classical approaches to the problem rely on using algebraic techniques to solve for these unknowns given two or more images. More recently, a batch analysis of image streams (the temporal tracks of distinguishable image features) under orthography has resulted in highly accurate reconstructions. We generalize this approach to perspective projection and partial or uncertain tracks by using a non-linear least squares technique. While our approach requires iteration, it quickly converges to the desired solution, even in the absence of a priori knowledge about the shape or motion. Important features of the algorithm include its ability to handle partial point tracks, to use line segment matches and point matches simultaneously, and to use an object-centered representation for faster and more accurate structure and motion recovery. We also show how a projective (as opposed to scaled rigid) structure can be recovered when the camera calibration parameters are unknown.

Point Cloud Data Fusion for Enhancing 2D Urban Flood Modelling

“…It also used to improve the accuracy, minimise projection errors from camera tracking, and refined the camera positions in order to extract a low-density or 'sparse' SfM point cloud data. In this stage, keypoint correspondences place constraints on camera pose orientation, which are reconstructed using a similarity transformation, whereas minimising errors is achieved by using a nonlinear least squares solution (Szeliski & Kang, 1994;Nocedal & Wright, 1999). Finally, triangulation is used to estimate the 3D point positions and incrementally reconstruct scene geometry, fixed into a relative coordinate system.…”

Automatic Pose Estimation of Uncalibrated Multi-View Images Based on a Planar Object with a Predefined Contour Model

“…The unknown parameters of the bundle adjustment procedure contain space feature points (including Types I and II) and camera parameters. The constraint of the bundle adjustment procedure is the collinearity condition described in Equation (9), and this constraint is often used in computer vision.…”

“…In the late 1980s, effective SfM techniques were developed, which aimed to reconstruct the unknown 3D scene structure and the camera positions and orientations from a set of feature correspondences simultaneously. Longuet-Higgins introduced a still widely-used two-frame relative orientation technique in 1981 [6]; however, the development of a multi-frame structure in motion techniques, including factorization methods [7], and global optimization techniques [8][9][10] was delayed. In 2004, Nister matched small subsets of images to one another and then merged them for a complete 3D reconstruction in the form of sparse point clouds [11].…”

Rigidity checking of 3D point correspondences under perspective projection

“…The basic nonlinear algorithm was developed earlier [9, lo]. Szeliski and Kang have recently and independently developed an algorithm for structure-from-motion which is similar to our method [12]. Their application is not the same as the one described here, however, in that they are seeking to recover object structure from many views and correspondences.…”