Lift-Off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR

Abstract

Three-dimensional modeling has long been regarded as an ideal application for virtual reality (VR), but current VR-based 3D modeling tools suffer from two problems that limit creativity and applicability: (1) the lack of control for freehand modeling, and (2) the difficulty of starting from scratch. To address these challenges, we present Lift-Off, an immersive 3D interface for creating complex models with a controlled, handcrafted style. Artists start outside of VR with 2D sketches, which are then imported and positioned in VR. Then, using a VR interface built on top of image processing algorithms, 2D curves within the sketches are selected interactively and "lifted" into space to create a 3D scaffolding for the model. Finally, artists sweep surfaces along these curves to create 3D models. Evaluations are presented for both long-term users and for novices who each created a 3D sailboat model from the same starting sketch. Qualitative results are positive, with the visual style of the resulting models of animals and other organic subjects as well as architectural models matching what is possible with traditional fine art media. In addition, quantitative data from logging features built into the software are used to characterize typical tool use and suggest areas for further refinement of the interface.

Screwing assembly oriented interactive model segmentation in HMD VR environment

“…In recent CavePainting and TiltBrush systems, users can create colorful art productions. Both “Drawing on Air” and “Lift‐Off” study intuitive 3D curve inputs, and the latter also allows users to import a reference image. In the work of Otsuki et al, visual and haptic feedback is used to provide the sensation of painting on virtual three‐dimensional objects using the MAI Painting Brush++.…”

Virtual reality inspection and painting with measured BRDFs

“…Users noted the bulkiness of the controller compared to real-life painting tools such as a brush or a pencil. We accept this limitation of the system, hoping for smaller, lighter, or more customizable solutions in the future, like the stylus presented by Jackson and Keefe [2016].…”

Brush2Model: Convolution surface‐based brushes for 3D modelling in head‐mounted display‐based virtual environments

“…In the artist‐oriented CavePainting system, users can create separate colorful brushes such as Tilt Brush. The works “Drawing on Air" and “Lift‐Off" focus on convenient 3D curve input approaches and specifically the “Lift‐Off" introduces the excellent idea of importing a hand‐drawn paper with pencil sketches into immersive VR for reference. Inspired by the above‐mentioned works, in this paper, we propose a more efficient immersive skeletal brush‐based modelling system, in which 3D smooth models can be easily created and fabricated due to its manifoldness of the adopted skeleton‐based implicit surface representation.…”

“…In recent years, virtual reality (VR) has thrived rapidly and has achieved lots of success in entertainment, military, and engineering. The consequent immersive 3D modelling in VR places users within the same world as the produced models, which allows for more freedom than sketch‐based interface. Google's Tilt Brush is a new VR app that allows users to paint in 3D space.…”