The evaluation of hand-crafted and learned-based features in Terrestrial Laser Scanning-Structure-from-Motion (TLS-SfM) indoor point cloud registration: the case study of cultural heritage objects and public interiors

Abstract

Modern technologies are commonly used to inventory different architectural or industrial objects (especially cultural heritage objects and sites) to generate architectural documentation or 3D models. The Terrestrial Laser Scanning (TLS) method is one of the standard technologies researchers investigate for accurate data acquisition and processing required for architectural documentation. The processing of TLS data to generate high-resolution architectural documentation is a multi-stage process that begins with point cloud registration. In this step, it is a common practice to identify corresponding points manually, semi-manually or automatically. There are several challenges for the TLS point cloud processing in the data registration process: correct spatial distribution, marking of control points, automation, and robustness analysis. This is particularly important when large, complex heritage sites are investigated, where it is impossible to distribute marked control points. On the other hand, when orientating multi-temporal data, there is also the problem of corresponding reference points. For this reason, it is necessary to use automatic tie-point detection methods. Therefore, this article aims to evaluate the quality and completeness of the TLS registration process using 2D raster data in the form of spherical images and Affine Hand-crafted and Learned-based detectors in the multi-stage TLS point cloud registration as test data; point clouds were used for the historic 17th-century cellars of the Royal Castle in Warsaw without decorative structures, two baroque rooms in the King John III Palace Museum in Wilanów with decorative elements, ornaments and materials on the walls and flat frescoes, and two modern test fields, narrow office, and empty shopping mall. The extended Structure-from-Motion was used to determine the tie points for the complete TLS registration and reliability analysis. The evaluation of detectors demonstrates that for the test sites exhibiting rich textures and numerous ornaments, a combination of AFAST, ASURF, ASIFT, SuperGlue and LoFTR can be effectively employed. For the point cloud registration of less textured buildings, it is advisable to use AFAST/ASIFT. The robust method for point cloud registration exhibits comparable outcomes to the conventional target-based and Iterative Closest Points methods.

Evaluation of 2D affine — hand-crafted detectors for feature-based TLS point cloud registration

“…The article aims to show the potential applications, together with their constraints, of the use of detectors and their a ne variants in the automatic TLS point cloud registration. A ne detector Investigations presented in this article are a continuation of the work presented in Markiewicz et al (2023), which proposed the TLS-SfM approach of TLS data registration for three a ne detectors (AFAST, ASIFT and ASURF). The present research aims to highlight that the choice of detector or a ne detector a ects data registration completeness, computation time, and registration quality.…”

“…Therefore, it is impossible to use this algorithm directly, and it is necessary to modify the input data and the Geometric Veri cation step. A detailed description of the modi ed TLS-SfM method is provided in Markiewicz et al (2023), and how the above steps can be modi ed is explained in sections 2.1 and 2.2.…”

“…The results of this investigation are presented in Figure 5. To interpret the results, the registration results were colour-coded according to the methodology presented in Markiewicz et al (2023): (1) green -the complete registration with the RMSE X , RMSE Y and RMSE Z ≤ 0.005 m and covariance factor > 0.5; (2) orange -preliminary registration obtained parameters should be treated as the initial parameters for Iterative Closest Point (ICP) registration and…”

“…Relations between spherical coordinates and coordinates on spherical images. (a) Graphical representation of the relations between polar coordinates measured and the raster image in spherical projection, (b) formula for recalculation of polar coordinates to spherical projection of arbitrary radius r, and (c) formula for recalculation of x, y spherical projection onto polar coordinates (Markiewiczand Zawieska, 2019;Markiewicz et al, 2023)…”