End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scans

SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans

“…Inspired by these attributes of artist-created 3D content, we aim to construct a CAD-based scene representation of an input RGB-D scan, with objects represented by individual CAD models and scene layout represented by lightweight meshes. In contrast to previous approaches which have individually tackled the tasks of CAD model alignment [21,2,3] and of layout estimation [27,24,6], we observe that object arrangement is typically tightly correlated with the scene layout. We thus propose to jointly optimize for CAD model alignment and scene layout to produce a globally-consistent CAD-based representation of the scene.…”

“…Avetisyan et al [2] proposed a scan-to-CAD retrieval and alignment approach leveraging learned features to detect objects in a 3D scan and establish correspondences across the domain gap of scan and CAD. They later built upon this work to develop a fully end-to-end trainable approach for this CAD alignment task [3]. For such approaches, each object is considered independently, whereas our approach exploits contextual information from object-object and object-layout to produce globally consistent CAD model alignment and layout estimation.…”

“…We follow the CAD alignment evaluation metric proposed by [2], which measures alignment accuracy where an alignment is considered successful if it falls within 20cm, 20 • , and 20% scale of the ground truth. On both SUNCG and ScanNet scans we compare to several state-of-the-art handcrafted geometric feature matching approaches [30,33,21] and learned approaches [39,2,3]. We additionally show qualitative comparisons in Figures 6 and 5.2.…”