CAD-Deform: Deformable Fitting of CAD Models to 3D Scans

Abstract

Shape retrieval and alignment are a promising avenue towards turning 3D scans into lightweight CAD representations that can be used for content creation such as mobile or AR/VR gaming scenarios. Unfortunately, CAD model retrieval is limited by the availability of models in standard 3D shape collections (e.g., ShapeNet). In this work, we address this shortcoming by introducing CAD-Deform 1 , a method which obtains more accurate CAD-to-scan fits by non-rigidly deforming retrieved CAD models. Our key contribution is a new non-rigid deformation model incorporating smooth transformations and preservation of sharp features, that simultaneously achieves very tight fits from CAD models to the 3D scan and maintains the clean, high-quality surface properties of hand-modeled CAD objects. A series of thorough experiments demonstrate that our method achieves significantly tighter scan-to-CAD fits, allowing a more accurate digital replica of the scanned real-world environment while preserving important geometric features present in synthetic CAD environments.

SMA-Net: Deep learning-based identification and fitting of CAD models from point clouds

“…[3,26,27] use similar way to align the CAD template with the 3D input. Then, to improve surface reconstruction accuracy, Vladislav et al [30] proposed to deform retrieved CAD models after their alignment to scans. However, this mesh deformation based method can only achieve rough approximations, and it results in changes in the geometric properties of the CAD model that is transformed in a mesh.…”

RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction

“…Object shapes are retrieved with CAD models or primitives with non-linear optimization. After that, 3D deep learning reforms this process into a learnable manner that model retrieval can be replaced with deep feature matching [3,4,5,29]. Semantic modeling presents delicate shape models though, the matching similarity and inference efficiency directly rely on the CAD dataset scale.…”

“…However, the expensive 3D convolutions in the scene level make them suffer from the resolution problem. Shape retrieval [3,4,5,29] provides an alternative method to pre-1 arXiv:2011.14744v1 [cs.CV] 30 Nov 2020 dict shapes by searching for a CAD model as similar to the incomplete object as possible. However, the accuracy and the computation efficiency depends on the model dataset scale.…”

A Revisit of Shape Editing Techniques: From the Geometric to the Neural Viewpoint

“…For example, Ovsjanikov et al [239] explored a shape dataset through the deformations of a template shape which abstracts the shape structure using several boxes. Ishimtsev et al [240] proposed a data-driven mesh deformation method, named CAD-Deform, to fit the retrieved synthetic CAD models to the real 3D scan.…”