Sketch‐to‐Design: Context‐Based Part Assembly

Abstract

Designing 3D objects from scratch is difficult, especially when the user intent is fuzzy and lacks a clear target form. We facilitate design by providing reference and inspiration from existing model contexts. We rethink model design as navigating through different possible combinations of part assemblies based on a large collection of pre-segmented 3D models. We propose an interactive sketch-to-design system, where the user sketches prominent features of parts to combine. The sketched strokes are analysed individually, and more importantly, in context with the other parts to generate relevant shape suggestions via adesign galleryinterface. As a modelling session progresses and more parts get selected, contextual cues become increasingly dominant, and the model quickly converges to a final form. As a key enabler, we use pre-learned part-based contextual information to allow the user to quickly explore different combinations of parts. Our experiments demonstrate the effectiveness of our approach for efficiently designing new variations from existing shape collections.

Data-driven segmentation and labeling of freehand sketches

“…We refer the interested reader to insightful discussions on sketch-based 3D modeling (e.g., constructive [Rivers et al 2010] or template-based [Kraevoy et al 2009;Olsen et al 2009]), sketch recognition (e.g., [Eitz et al 2012a]), sketch-based shape retrieval (e.g., [Eitz et al 2012b]). Being able to segment sketches, at the part or object level, greatly benefit applications like sketch-based object modeling [Lee and Funkhouser 2008;Xie et al 2013] and scene modeling [Shin and Igarashi 2007;Xu et al 2013]. Such segmentations are typically achieved by design (e.g., manually drawing individual logical components in different layers, or drawing components one by one in a specific order), which however is often an error-prone process even 1(a) 2(a) 3(a) 4(a) Figure 2: The first row shows 4 input sketches to our system, with strokes color-coded.…”

“…Assembly-based 3D modeling was pioneered by Funkhouser et al [2004] and has been extended for data-driven modeling suggestions [Chaudhuri et al 2011] and open-ended 3D modeling [Kalogerakis et al 2012;Xu et al 2012]. Sketch-based interfaces have also been explored for object or scene compositions [Shin and Igarashi 2007;Lee and Funkhouser 2008;Xie et al 2013], which, however, require manual sketch segmentation, as discussed previously. Shen et al [2012] present a part-assembly approach for recovering 3D geometry and structure from raw scans acquired by RGBD cameras.…”

Juxtapoze

“…A recent example of a digital drawing aid is Vignette [19], which supports drawing by automatically filling the drawing canvas with pen-and-ink textures based on user input. Recently, there has be great interest in 3D shape assembly using sketch based search [34] constrainted by contextual information of component parts Methods which use sketch based retrieval [9,10] are targeted towards minimizing the number of irrelevant objects rather than maximizing the number of possibly relevant objects. This does not support serendipitous discoveries [24].…”

Sketch2Scene

“…Context-based search has started to show its importance in 3D computer graphics only in recent years, e.g., [Fisher and Hanrahan 2010], which however assumes the availability of explicit context, which is missing in our problem. Inspired by the recent works on data-driven suggestions [Chaudhuri and Koltun 2010;Fisher and Hanrahan 2010;Lee et al 2011;Fisher et al 2011], Xie et al [2012] apply context-based search to interactive sketch-based assembly of 3D parts [Lee and Funkhouser 2008]. However, their technique also requires explicit context.…”

“…The existing techniques for sketch-based search and composition of 3D models [Shin and Igarashi 2007;Lee and Funkhouser 2008;Xie et al 2012] typically repeat the following process for individual desired models: first 2D sketch, then retrieval and finally 3D placement. This iterative 2D-3D-2D process is not compatible with the traditional design workflow (i.e., 2D concept design followed by 3D modeling), which is largely sequential and often demands different professionals specialized for different tasks (e.g., concept artists, 3D modelers) [Chopine 2011].…”