Method for registration of 3D shapes without overlap for known 3D priors

Abstract

In 3D registration of point clouds, the goal is to find an optimal transformation that aligns the input shapes, provided that they have some overlap. Existing methods suffer from performance degradation when the overlapping ratio between the neighbouring point clouds is small. So far, there is no existing method that can be adopted for aligning shapes with no overlap. In this letter, to the best of knowledge, the first method for the registration of 3D shapes without overlap, assuming that the shapes correspond to partial views of a known semi-rigid 3D prior is presented. The method is validated and compared to existing methods on FAUST, which is a known dataset used for human body reconstruction. Experimental results show that this approach can effectively align shapes without overlap. Compared to existing state-of-theart methods, this approach avoids iterative optimization and is robust to outliers and inherent inaccuracies induced by an initial rough alignment of the shapes. Introduction: 3D registration is a classical and fundamental problem for countless applications. Since commodity depth cameras become less expensive and more accurate, depth images play an increasingly important role in numerous tasks [1]. In order to obtain comprehensive information from 3D scenery, point clouds captured from multiple views need to be aligned. The well-established method is iterative closest point (ICP) [2] based on which a myriad of flavours have been proposed. In ICP, given a source shape and a target shape, the following steps are performed: (1) for each point in the source shape, identify the closest corresponding point in the target shape; (2) predict the transformation by minimizing the mean square Euclidean distance between these correspondences; (3) transform the source shape using the predicted transformation from step 2; (4) iterate the above steps until the mean square distance reaches a pre-defined threshold. ICP and its variants are the dominating methods for the task of 3D registration. However, ICP-based methods assume that the source and target shapes have been roughly aligned with a sufficient overlap.. Deep learning has shown its excellent ability to solve various problems which are difficult or impossible to address using traditional approaches. Recent research strives to explore 3D registration via deep learning [3], [4], [5], [6]. However, these methods are designed for shapes that partially overlap. In this letter, we present a novel deep learning method for 3D shape registration. Compared to the existing methods, the main advantage of our method is that we successfully handled the non-overlapping shape registration problem. We assume that the shapes correspond to partial views of a known semi-rigid 3D prior. This problem is impossible to be addressed using ICP due to the lack of point correspondences. This is addressed in this letter of which the main contributions can be summarized as follows:

AlignBodyNet: Deep Learning-Based Alignment of Non-Overlapping Partial Body Point Clouds From a Single Depth Camera

“…However, the two outputs can have large shape and pose variations. We argue there are two main reasons for this phenomenon: (i) in [27] that authors predicted complete body shapes from single partial point clouds, which is an illposed problem; and (ii) for the same subject, the body shape should be the same no matter if it is being observed from front-or back-facing views (our third insight).…”

“…The closest work to ours is the recently published method in [27]. It predicted two completed bodies for registration from two partial point clouds, respectively.…”

“…Intuitively, this approach fails for our task because no correspondences between the front-and backfacing partial body point clouds exist due to the lack of an overlap area between the scans. Inspired by the work of [27], we address this task by converting it to a problem of finding virtual correspondences, and rewrite equation ( 1):…”

“…Therefore, shape-interrelated features require to be interpreted to parametric body vertices. MLP is used to this end in [27]. However, the two outputs can have large shape and pose variations.…”

“…[33], [16]), we use the same MLP decoder as in [27] in order to validate our idea and to fairly compare our method with the work of [27]. Main conceptual improvements over [27] include (i) a two-stream decoder architecture, and (ii) we add a Transformer that transforms τ (X) from the front-facing view-based coordinate to the back-facing view-based coordinate by the groundtruth transformation. This Transformer plays an important role in our study as we can enforce a powerful constraint τ (X) = ζ (Y ).…”