End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scans

Abstract

Figure 1: From a 3D scan and a set of CAD models, our method learns to predict 9DoF CAD model alignments to the objects of the scan, in a fully-convolutional, end-to-end fashion. Our proposed 3D CNN first detects objects in the scan, then uses the regressed object bounding boxes to establish symmetry-aware object correspondences between a scan object and CAD model, which inform our differentiable Procrustes alignment loss, enabling learning of alignment-informed correspondences and producing CAD model alignment to a scan in a single forward pass. AbstractWe present a novel, end-to-end approach to align CAD models to an 3D scan of a scene, enabling transformation of a noisy, incomplete 3D scan to a compact, CAD reconstruction with clean, complete object geometry. Our main contribution lies in formulating a differentiable Procrustes alignment that is paired with a symmetry-aware dense object correspondence prediction. To simultaneously align CAD models to all the objects of a scanned scene, our approach detects object locations, then predicts symmetry-aware dense object correspondences between scan and CAD geometry in a unified object space, as well as a nearest neighbor CAD model, both of which are then used to inform a differentiable Procrustes alignment. Our approach operates in a fullyconvolutional fashion, enabling alignment of CAD models to the objects of a scan in a single forward pass. This enables our method to outperform state-of-the-art approaches by 19.04% for CAD model alignment to scans, with ≈ 250× faster runtime than previous data-driven approaches.

Uncertainty Estimation in Deep Neural Networks for Point Cloud Segmentation in Factory Planning

“…The framework is called Scan2CAD, and it describes a frequentist deep neural network that consumes voxelized point clouds as well as CAD models of eight household objects and directly learns the 6DoF CAD model alignment within the point cloud. The system that is presented in [19] has similar input data and estimates the 9DoF pose, i.e., translation, rotation and scale, of the same household objects. A framework for the alignment of CAD models, which is based on global descriptors computed by using the Viewpoint Feature Histogram approach [20] rather than neural networks, is discussed in [21].…”

Globally Optimal Point Set Registration by Joint Symmetry Plane Fitting

“…In recent years, the exploitation of symmetry in shapes has become a popular topic in the fields of 3D vision and deep learning. It has been used in various works on 3D object completion [32,35,36], shape correspondence prediction [3,28,46] and 3D object detection [1,6,31]. For a comprehensive review of symmetry detection, we kindly refer the reader to Liu et al's review [23] and its applications [27].…”

“…If X and Y contain points in the 2D plane, R and t form an element of the group S E (2). If X and Y contain 3D points, R and t will be an element of S E (3). Note that alignment denotes a more general idea rather than just the minimization of the sum of distances between each point of Y and its closest point within X .…”

From a Point Cloud to a Simulation Model—Bayesian Segmentation and Entropy Based Uncertainty Estimation for 3D Modelling

“…The Scan2CAD framework directly estimates the 6DoF CAD model alignment, that is, position and rotation, of eight household objects on the basis of a voxelized point cloud and the respective CAD models [ 32 ]. An extension of this framework, which is capable of 9DoF pose estimation on the same data set, is discussed in [ 33 ]. Apart from deep neural networks a CAD model alignment framework based on global descriptors extracted by the Viewpoint Feature Histogram approach [ 34 ] is described in [ 35 ].…”