Structure from motion causally integrated over time

Abstract

ÐWe describe an algorithm for reconstructing three-dimensional structure and motion causally, in real time from monocular sequences of images. We prove that the algorithm is minimal and stable, in the sense that the estimation error remains bounded with probability one throughout a sequence of arbitrary length. We discuss a scheme for handling occlusions (point features appearing and disappearing) and drift in the scale factor. These issues are crucial for the algorithm to operate in real time on real scenes. We describe in detail the implementation of the algorithm, which runs on a personal computer and has been made available to the community. We report the performance of our implementation on a few representative long sequences of real and synthetic images. The algorithm, which has been tested extensively over the course of the past few years, exhibits honest performance when the scene contains at least 20-40 points with high contrast, when the relative motion is ªslowº compared to the sampling frequency of the frame grabber (QHHz), and the lens aperture is ªlarge enoughº (typically more than QH o of visual field).

Monocular Model-Based 3D Tracking of Rigid Objects: A Survey

“…Pose and structure can also be recursively estimated using the Extended Kalman filter [5,12,22,63,28] discussed in Subsection 2.6.1. In particular, [28] shows that it can yield very good results in real-time.…”

KALMANSAC: robust filtering by consensus

“…Although this integral is computationally expensive, it can be approximated by setting p(x t |y t ,χ t−1 ) = δ(x t − x second-order random walk that fits equation (1), and structure from motion, where we adopt a model borrowed from [2,4], which is non-linear. In the latter case, the computation of the MAP densities are approximated by an extended Kalman filter.…”

“…We attribute part of this failure to the lack of availability of suitable robust inference techniques that can be applied in causal data processing (there are a few exceptions, upon which we will comment later.) Note that batch-processing based SFM algorithms, together with the associated techniques for handling outliers, cannot be directly applied on these problems as they introduce destabilizing delays in the feedback loop [4]. On the other hand, existing robust filtering techniques, which we review in Section 1.1, either cannot tolerate a large proportion of outliers, or are not suitable for real-time implementation.…”

State estimation and control for systems with perspective outputs

“…We are interested here in filtering-like or iterative algorithms that continuously improve the estimates as more data (i.e., images) are acquired and that are robust with respect to measurement noise. Following the lead of other research (cf., e.g., [11,8,15,9,14,3] and references therein) we formulate this as a state-estimation problem and utilize the estimator derived in Section 2 to solve it. One of the main contributions of this paper is that-opposite what happens with most previous algorithms-the one proposed here is globally convergent provided that suitable observability assumptions are satisfied.…”