Interactive 3D Model Acquisition and Tracking of Building Block Structures

Abstract

Fig. 1. An overhead view of our system in action. A depth camera observes the creation of a physical structure made of building blocks. Our system continuously updates a virtual model of the structure as the user adds and removes pieces.Abstract-We present a prototype system for interactive construction and modification of 3D physical models using building blocks. Our system uses a depth sensing camera and a novel algorithm for acquiring and tracking the physical models. The algorithm, Lattice-First, is based on the fact that building block structures can be arranged in a 3D point lattice where the smallest block unit is a basis in which to derive all the pieces of the model. The algorithm also makes it possible for users to interact naturally with the physical model as it is acquired, using their bare hands to add and remove pieces. We present the details of our algorithm, along with examples of the models we can acquire using the interactive system. We also show the results of an experiment where participants modify a block structure in the absence of visual feedback. Finally, we discuss two proof-of-concept applications: a collaborative guided assembly system where one user is interactively guided to build a structure based on another user's design, and a game where the player must build a structure that matches an on-screen silhouette.Index Terms-Interactive physical model building, 3D model acquisition, object tracking, depth cameras, building block structures.

The effectiveness of an AR-based context-aware assembly support system in object assembly

“…A depth camera-based system using an algorithm called LatticeFirst [14] is used for real-time acquisition and tracking of LEGO blocks. It is fast and effective, providing users with the ability to incrementally construct a block-based physical model using their bare hands while the system updates the model's virtual representation in real-time.…”

“…Recently, however, context-aware on-line error detecting AR systems have attracted attention [14]. This enables us to evaluate the potential of augmented reality in a more comprehensive way.…”

A review and framework of laser-based collaboration support

“…Better technology is needed for indoor/outdoor use, (e.g., Miller, White, Charbonneau, Kanzler, & LaViola, 2012): Monocular cameras using structured light are relatively inexpensive, reliable and do not require calibration. While some cameras deliver satisfactory performance indoors using IR structured light, other devices use stereoscopies (Kim, Lee, Yang, Kwak, & Kim, 2012), or rely on laser structured projections to deliver depth information (Chavez & Karstoft, 2012).…”

DuploTrack

“…In a contemporaneous work, Miller et al [18] solve a similar problem of tracking and inferring how a Duplo R block model is built. They assume that the model always stays with its base on the table to reduce the tracking to 3 degrees of freedom (DOF): two for in-plane translation and one for inplane rotation.…”