Real-Time Large-Scale Dense 3D Reconstruction with Loop Closure

Abstract

In the highly active research field of dense 3D reconstruction and modelling, loop closure is still a largely unsolved problem. While a number of previous works show how to accumulate keyframes, globally optimize their pose on closure, and compute a dense 3D model as a post-processing step, in this paper we propose an online framework which delivers a consistent 3D model to the user in real time. This is achieved by splitting the scene into submaps, and adjusting the poses of the submaps as and when required. We present a novel technique for accumulating relative pose constraints between the submaps at very little computational cost, and demonstrate how to maintain a lightweight, scalable global optimization of submap poses. In contrast to previous works, the number of submaps grows with the observed 3D scene surface, rather than with time. In addition to loop closure, the paper incorporates relocalization and provides a novel way of assessing tracking quality.

Real-time Scalable Dense Surfel Mapping

“…The reconstruction accuracy is defined as the mean difference between the reconstructed model and the ground truth model. Here, we compare the proposed mapping method with BundleFusion [7], ElasticFusion [8], InfiniTAM [13] and the recently published FlashFusion [16]. To evaluate the ability to maintain the global consistency, we also evaluate Ours w/o loop in which the loop closure in ORB-SLAM2 is disabled.…”

Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence

“…Furthermore, the replacement of the TSDF representation by a highresolution binary voxel grid has also been considered by Reichl et al [25] to improve the scalability and reduce the memory requirements. Recent extensions include the detection of loop closures [2,11,16] to reduce drift artifacts in camera localization as well as multi-client collaborative acquisition and reconstruction of static scenes [6]. This progress in real-time capturing enabled the development of various telepresence applications.…”

“…3). Here, the surface is represented in terms of implicit truncated signed distance fields (TSDFs) and stored as a sparse unordered set of voxel blocks using spatial hashing [2,11,12,23,25]. Input to the reconstruction pipeline is an incremental stream of RGB-D images which is processed in an online fashion.…”

“…tracking [9,12,21,23,25,29,31] (as also used in the SLAMCast system) or using bundle adjustment for globally-consistent reconstruction [2,11]. Using this pose, non-visible voxel block data are streamed out to CPU memory whereas visible blocks in CPU memory are streamed back into GPU memory [12,16,23,25].…”

“…Note that this downsampling is only performed during allocation whereas the whole depth image is still used for data fusion to employ TSDF-based regularization. In the context of globally-consistent 3D reconstruction using bundle-adjusted submaps [6,11], our filter improves the compactness of the respective submap into which the RGB-D data Algorithm 1 Our optimized server voxel block data integration Input: Received TSDF voxel block positions P T SDF and voxel data V T SDF Output: Voxel block position list P MC for updating the stream sets 1:…”