Comprehensive Use of Curvature for Robust and Accurate Online Surface Reconstruction

Abstract

Interactive real-time scene acquisition from hand-held depth cameras has recently developed much momentum, enabling applications in ad-hoc object acquisition, augmented reality and other fields. A key challenge to online reconstruction remains error accumulation in the reconstructed camera trajectory, due to drift-inducing instabilities in the range scan alignments of the underlying iterative-closest-point (ICP) algorithm. Various strategies have been proposed to mitigate that drift, including SIFT-based pre-alignment, color-based weighting of ICP pairs, stronger weighting of edge features, and so on. In our work, we focus on surface curvature as a feature that is detectable on range scans alone and hence does not depend on accurate multi-sensor alignment. In contrast to previous work that took curvature into consideration, however, we treat curvature as an independent quantity that we consistently incorporate into every stage of the real-time reconstruction pipeline, including densely curvature-weighted ICP, range image fusion, local surface reconstruction, and rendering. Using multiple benchmark sequences, and in direct comparison to other state-of-the-art online acquisition systems, we show that our approach significantly reduces drift, both when analyzing individual pipeline stages in isolation, as well as seen across the online reconstruction pipeline as a whole.

SurfelMeshing: Online Surfel-Based Mesh Reconstruction

“…27: if SD(p) = 0 and I s (p) = −1 then for all pixels q in p's 8-neighborhood do 33: if I(q) = i − 1 then sum += δ(q); count += 1 34: # Apply the (weighted) averaged delta? 35: if count > 0 then 36:…”

“…11. Cut through surfaces from ElasticFusion (top left) and [36] (bottom left) compared to corresponding surfels in our approach (right). Our method generates smoother surfaces due to our denoising steps (Sec.…”

PSDF Fusion: Probabilistic Signed Distance Function for On-the-fly 3D Data Fusion and Scene Reconstruction

“…Instead of utilizing volumetric spatial representations, [13] proposes point-based fusion that maintains light-weight dense point cloud or surfels as 3D maps. Elas-ticFusion [31] and InfiniTAM v3 [23] are efficient implementations of [13] with several extensions; [16] further improves [13] by introducing surface curvatures. The point-based methods are unable to output mesh online, hence may not be suitable for physics-based applications.…”

CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras

“…With the advent of Kinectstyle active depth sensors, the KinectFusion [1] algorithm permits dense volumetric reconstruction of the scene in realtime, enabling mesh models output for physics-based augmented reality (AR) [24] and 3D printing [25]. Improved frameworks have then been proposed in the aspects of memory efficiency [26]- [28], large space representation [8], [11], [27], [29], camera trajectory accuracy with loop closure detection and optimization [12], [27], [30], and scene representation such as surfels [31] or hybrid data structure [32].…”

“…The SDF represents the surface interfaces implicitly as zeros, and the mesh models can be extracted using the marching cubes type algorithm [4]. Instead of volumes, surfels [10], [31], [43], [44] are also exploited to represent the scene. It renders the scene with the surface-splatting technique [45] and reduces the computational complexity and memory overhead compared with the volumetric approaches.…”