Fast and Accurate Structure and Motion Estimation

Abstract

Abstract. This paper describes a system for structure-and-motion estimation for real-time navigation and obstacle avoidance. We demonstrate a technique to increase the efficiency of the 5-point solution to the relative pose problem. This is achieved by a novel sampling scheme, where we add a distance constraint on the sampled points inside the RANSAC loop, before calculating the 5-point solution. Our setup uses the KLT tracker to establish point correspondences across time in live video. We also demonstrate how an early outlier rejection in the tracker improves performance in scenes with plenty of occlusions. This outlier rejection scheme is well suited to implementation on graphics hardware. We evaluate the proposed algorithms using real camera sequences with fine-tuned bundle adjusted data as ground truth. To strenghten our results we also evaluate using sequences generated by a state-of-the-art rendering software. On average we are able to reduce the number of RANSAC iterations by half and thereby double the speed.Structure and motion (SaM) estimation from video sequences is a well explored subject [1][2][3]. The underlying mathematics is well understood, see e.g. [1], and commercial systems, such as Boujou by 2d3 [4], are used in the movie industry on a regular basis. Current research challenges involve making such systems faster, more accurate, and more robust, see e.g. [2,3]. These issues are far from solved, as is illustrated by the 2007 DARPA urban challenge [5]. In the end, none of the finalists chose to use the vision parts of their systems, instead they relied soley on LIDAR to obtain 3D structure. Clearly there is still work to be done in the field. This paper aims to increase the speed and accuracy in structure-and-motion estimation for an autonomous system with a forward looking camera, see figure 1. Although on a smaller scale, this platform has the same basic geometry and motion patterns as the DARPA contenders, and as the vision based collision warning systems developed for automotive applications. In such systems, estimated 3D structure can be used to detect obstacles and navigable surfaces.When dealing with forward motion there are a number of problems that must be adressed. The effective baseline is on average much smaller than for the sideways motion case, resulting in a more noise sensitive structure estimation. A tracked point feature near the camera often has a short lifespan because it 2 Johan Hedborg, Per-Erik Forssén, and Michael Felsberg quickly moves out of the the visual field. Unfortunately, such points also contain most of the structural information [6]. Forward motion also produces large scale changes in some parts of the image, and this can be a problem for some trackers.This paper studies the calibrated SaM formulation, which has several advantages over the uncalibrated formulation. In calibrated SaM, estimated cameras and structure will be in Euclidean space instead of a projective space, and we can use more constrained problem formulations [1]. Planar-dominant scenes are not ...

Robust stereo visual odometry from monocular techniques

“…This approach simplifies the fusion of ancillary sensors such as IMUs, but complicates the process to revert associations retroactively [20], [21], [22], [23]. The main alternative to this approach is to instead rely on a bundle adjustment step to refine the map and camera poses by minimizing the total reprojection error [24], [4], [11], [5].…”

“…These matches are then used to compute the egomotion by the means of essential matrix estimation [11], PNP [4], [1] or the trifocal tensor [12]. Direct methods on the other hand operate on the whole images thus producing a dense or semi-dense motion field [13], [14], [15], [16].…”

Fast iterative five point relative pose estimation

“…When tracking frame-by-frame, the changes in viewing angle and scale are sufficiently small for tracking to work well. In order to improve stability of tracked points, the track-retrack (crosscheck) has been applied in certain cases, [6]. All these steps were performed using Willow Garage's OpenCV implementations.…”

Gyroscope-based video stabilisation with auto-calibration

“…To improve the quality of the tracks, we perform trackretrack [21], i.e. we track both forwards and backwards in the slices, and only keep those tracks that were successfully retracked to within 0.5 pixels of their initial positions.…”