2D-3D fusion for layer decomposition of urban facades

Abstract

We present a method for fusing two acquisition modes, 2D photographs and 3D LiDAR scans, for depth-layer decomposition of urban facades. The two modes have complementary characteristics: point cloud scans are coherent and inherently 3D, but are often sparse, noisy, and incomplete; photographs, on the other hand, are of high resolution, easy to acquire, and dense, but view-dependent and inherently 2D, lacking critical depth information. In this paper we use photographs to enhance the acquired LiDAR data. Our key observation is that with an initial registration of the 2D and 3D datasets we can decompose the input photographs into rectified depth layers. We decompose the input photographs into rectangular planar fragments and diffuse depth information from the corresponding 3D scan onto the fragments by solving a multi-label assignment problem. Our layer decomposition enables accurate repetition detection in each planar layer, using which we propagate geometry, remove outliers and enhance the 3D scan. Finally, the algorithm produces an enhanced, layered, textured model. We evaluate our algorithm on complex multi-planar building facades, where direct autocorrelation methods for repetition detection fail. We demonstrate how 2D photographs help improve the 3D scans by exploiting data redundancy, and transferring high level structural information to (plausibly) complete large missing regions.

Automatic Removal of Imperfections and Change Detection for Accurate 3D Urban Cartography by Classification and Incremental Updating

“…Large holes in the background layer, caused by occlusions from foreground layer objects, are then filled by planar or horizontal interpolation. However, such an approach may result in false features in case of insufficient repetitions or lack of symmetry [17]. In our work, we aim to resolve this problem by using a multi-sessional approach in which multiple scans of the same environment obtained on different days and at different times of the day are matched and used to complete the occluded regions.…”

Online facade reconstruction from dominant frequencies in structured point clouds

“…Leveraging databases of millions of images [4] completes scenes by finding semantically similar pictures. Fusing both 2D and 3D images [6] computes a layer decomposition from a registration between the 2D and 3D data sets which allows for outlier removal and geometry propagation. [14] introduces a context sensitive surface completion.…”

Coupled structure-from-motion and 3D symmetry detection for urban facades

“…Further, if the SfM outputs have gross errors due to ambiguity across repeated elements (see Figure 1), subsequent analysis of the 3D point sets can only extract wrong constraints. Other possibilities involve integrating information from other acquisition modes (e.g., LiDAR scans as used by Zheng et al [2010] and Li et al [2011]) or allowing the user to indicate symmetry structures on 3D data (e.g., [Nan et al 2010]) (see the recent survey of Mitra et al [? ] for a more detailed discussion on the use of symmetry priors in architecture modeling).…”