Learning Part Generation and Assembly for Sketching Man‐Made Objects

Abstract

Modeling 3D objects on existing software usually requires a heavy amount of interactions, especially for users who lack basic knowledge of 3D geometry. Sketch‐based modeling is a solution to ease the modelling procedure and thus has been researched for decades. However, modelling a man‐made shape with complex structures remains challenging. Existing methods adopt advanced deep learning techniques to map holistic sketches to 3D shapes. They are still bottlenecked to deal with complicated topologies. In this paper, we decouple the task of sketch2shape into a part generation module and a part assembling module, where deep learning methods are leveraged for the implementation of both modules. By changing the focus from holistic shapes to individual parts, it eases the learning process of the shape generator and guarantees high‐quality outputs. With the learned automated part assembler, users only need a little manual tuning to obtain a desired layout. Extensive experiments and user studies demonstrate the usefulness of our proposed system.

DualShape: Sketch-Based 3D Shape Design With Part Generation and Retrieval

“…For assembly-based modeling, Li et al [18] proposed learning part generation and assembly for structural shape generation based on volume representation; however, their method is based on semantics and is not suitable for recovering shapes with complex structures. To break out of the issue, Du et al [3] decomposed the generation task into modeling and shape assembly based on parts. Based on the shape structure learning [25], [28] and the densely partitioned dataset [26].…”

SimpModeling: Sketching Implicit Field to Guide Mesh Modeling for 3D Animalmorphic Head Design

“…In recent years, learning-based solutions have been popular for sketch-based 3D shape generation and editing [10], [11], [15], [18], [19], [20], [38], [39], [58], [59], [60], [61], [62], [63], [64]. For example, Nishida et al [64] proposed inferring urban building parameters from freehand sketches with convolutional neural networks, while Huang et al [62] presented an interactive modeling system that infers parameters for procedural modeling from sketches.…”

“…However, the above parametric regression-based methods work only for 3D shapes within a specific category that can be easily parameterized. Du et al [63] adopted implicit learning to produce artificial object parts from sketches and proposed a deep regression model to predict the position of the parts, while Sketch2CAD [15] enables users to achieve controllable part-based CAD object modeling by sketching in context. SimpModeling [11] utilized a coarse-to-fine modeling scheme, allowing users to create desired animalmorphic heads with 3D curves and on-surface sketching.…”