Learning Transformation Synchronization

Abstract

Reconstructing the 3D model of a physical object typically requires us to align the depth scans obtained from different camera poses into the same coordinate system. Solutions to this global alignment problem usually proceed in two steps. The first step estimates relative transformations between pairs of scans using an off-the-shelf technique. Due to limited information presented between pairs of scans, the resulting relative transformations are generally noisy. The second step then jointly optimizes the relative transformations among all input depth scans. A natural constraint used in this step is the cycle-consistency constraint, which allows us to prune incorrect relative transformations by detecting inconsistent cycles. The performance of such approaches, however, heavily relies on the quality of the input relative transformations. Instead of merely using the relative transformations as the input to perform transformation synchronization, we propose to use a neural network to learn the weights associated with each relative transformation. Our approach alternates between transformation synchronization using weighted relative transformations and predicting new weights of the input relative transformations using a neural network. We demonstrate the usefulness of this approach across a wide range of datasets.

Learning Multiview 3D Point Cloud Registration

“…Taking a step further, other modern methods make use of the global cycle-consistency and optimize only over the poses starting from an initial set of pairwise maps. This efficient approach is known as synchronization [10,61,2,56,3,5,43,69,7,35]. Global structurefrom-motion [17,70] aims to synchronize the observed relative motions by decomposing rotation, translation and scale components.…”

“…Probably the most similar work to ours is [35], where the authors aim to adapt the edge weights for the transformation synchronization layer by learning a data driven weighting function. A major conceptual difference to our approach is that relative transformation parameters are estimated using FPFH [54] in combination with FGR [69] and thus, unlike ours, are not learned.…”

“…A major conceptual difference to our approach is that relative transformation parameters are estimated using FPFH [54] in combination with FGR [69] and thus, unlike ours, are not learned. Furthermore, in each iteration [35] has to convert the point clouds to depth images as the weighting function is approximated by a 2D CNN. On the other hand our whole approach operates directly on point clouds, is fully differentiable and therefore facilitates learning a global, multiview point cloud registration in an end-to-end manner.…”

“…In current state-of-the-art [69,35,7] edges E of G are initialized with (noisy) relative transformation parameters {M ij }, obtained by an independent, auxiliary pairwise registration algorithm. Global scene consistency is enforced via a subsequent synchronization algorithm.…”

Tensor maps for synchronizing heterogeneous shape collections

“…Despite significant advances in shape matching (c.f. [van Kaick et al 2011) and particularly more recent works on matching a shape collection jointly to improve the maps between pairs of shapes (or map synchronization) [Chen et al 2014; Cosmo et al 2017; Huang et al 2014; Huang and Guibas 2013; Huang et al 2012, 2019; Kim et al 2012; Nguyen et al 2011; Wang et al 2013; Zhang et al 2018a], the outputs of state-of-the-art approaches remain insufficient for high-quality analysis of heterogeneous shape collections.…”