Sketch2Scene

Abstract

Figure 1: Without any user intervention, our framework automatically turns a freehand sketch drawing depicting multiple scene objects (left) to semantically valid, well arranged scenes of 3D models (right). (The ground and walls were manually added.) AbstractThis work presents Sketch2Scene, a framework that automatically turns a freehand sketch drawing inferring multiple scene objects to semantically valid, well arranged scenes of 3D models. Unlike the existing works on sketch-based search and composition of 3D models, which typically process individual sketched objects one by one, our technique performs co-retrieval and co-placement of 3D relevant models by jointly processing the sketched objects. This is enabled by summarizing functional and spatial relationships among models in a large collection of 3D scenes as structural groups. Our technique greatly reduces the amount of user intervention needed for sketch-based modeling of 3D scenes and fits well into the traditional production pipeline involving concept design followed by 3D modeling. A pilot study indicates that it is promising to use our technique as an alternative but more efficient tool of standard 3D modeling for 3D scene construction.

ShapeFindAR: Exploring In-Situ Spatial Search for Physical Artifact Retrieval using Mixed Reality

“…Such novel user interfaces for search or design are an opportunity to enable widespread access to personalized artifacts, by lowering the skill floor while retaining high expressivity for more proficient users. Novel modalities are applicable both to searching for existing artifacts [20,49] and generating new ones [5,32,63]. ShapeFindAR can be considered to be a comparable approach to artifact retrieval, with an emphasis on coarse inputs that are -ideally -transferred to high-fidelity artifacts from a model repository.…”

Seg2pix: Few Shot Training Line Art Colorization with Segmented Image Data

“…Since sketch parsing analysis is one of the important topics in the multimedia community, several studies have contributed semantic parsing of freehand sketches [18,19]. Various methods of sketch parsing have been reported such as those focusing on sketch images [20][21][22] and 3D retrieval [23][24][25]. In this paper, we perform sketch parsing using a combination of algorithms: trap ball segmentation and Seg2pix network.…”

AR interior designer: Automatic furniture arrangement using spatial and functional relationships

“…Xu et al construct 3D in-house scenes by sketching the furniture and their arrangement [14]. Spatial relationships of different virtual objects are considered.…”

“…The limitation is that all possible configurations have to be given to the system. Xu also considered the hierarchical relationship [14]. However, these furniture arrangement methods did not encounter with the parameters of a real environment.…”