Bound constrained bundle adjustment for reliable 3D reconstruction

Abstract

Abstract:Bundle adjustment (BA) is a common estimation algorithm that is widely used in machine vision as the last step in a feature-based three-dimensional (3D) reconstruction algorithm. BA is essentially a non-convex non-linear least-square problem that can simultaneously solve the 3D coordinates of all the feature points describing the scene geometry, as well as the parameters of the camera. The conventional BA takes a parameter either as a fixed value or as an unconstrained variable based on whether the parameter is known or not. In cases where the known parameters are inaccurate but constrained in a range, conventional BA results in an incorrect 3D reconstruction by using these parameters as fixed values. On the other hand, these inaccurate parameters can be treated as unknown variables, but this does not exploit the knowledge of the constraints, and the resulting reconstruction can be erroneous since the BA optimization halts at a dramatically incorrect local minimum due to its non-convexity. In many practical 3D reconstruction applications, unknown variables with range constraints are usually available, such as a measurement with a range of uncertainty or a bounded estimate. Thus to better utilize these pre-known, constrained, but inaccurate parameters, a bound constrained bundle adjustment (BCBA) algorithm is proposed, developed and tested in this study. A scanning fiber endoscope (the camera) is used to capture a sequence of images above a surgery phantom (the object) of known geometry. 3D virtual models are reconstructed based on these images and then compared with the ground truth. The experimental results demonstrate BCBA can achieve a more reliable, rapid, and accurate 3D reconstruction than conventional bundle adjustment.

Feature-Based Three-Dimensional Registration for Repetitive Geometry in Machine Vision

“…Different with multiview stereo, SfM doesn't require camera information. It estimates all the camera parameters and 3D scene simultaneously by solving a non-linear, non-convex optimization problem [20]. With little camera baseline and therefore high depth uncertainty, the non-convexity of SfM may generate a completely unacceptable 3D model.…”

Axial-Stereo 3-D Optical Metrology for Inner Profile of Pipes Using a Scanning Laser Endoscope

“…Instead of using two images, each 3D point would have several projections in the multiple axial images, from which the estimation of the 3D points is more robust and accurate. Moreover, considering the inaccuracy of linear motion of the camera in practical applications, a bound constrained optimization algorithm can be implemented by utilizing the motion error as a priori knowledge [28]. …”

Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model

“…Bundle adjustment is often used as the last step in feature-based 3D reconstruction, following the prior steps of feature detection and feature matching. Once we have the 3D coordinates of the feature points and their projections in the 2D endoscopic image, a constrained bundle adjustment algorithm is achieved 9,10 .…”