Live Texturing of Augmented Reality Characters from Colored Drawings

Abstract

Abstract-Coloring books capture the imagination of children and provide them with one of their earliest opportunities for creative expression. However, given the proliferation and popularity of digital devices, real-world activities like coloring can seem unexciting, and children become less engaged in them. Augmented reality holds unique potential to impact this situation by providing a bridge between real-world activities and digital enhancements. In this paper, we present an augmented reality coloring book App in which children color characters in a printed coloring book and inspect their work using a mobile device. The drawing is detected and tracked, and the video stream is augmented with an animated 3-D version of the character that is textured according to the child's coloring. This is possible thanks to several novel technical contributions. We present a texturing process that applies the captured texture from a 2-D colored drawing to both the visible and occluded regions of a 3-D character in real time. We develop a deformable surface tracking method designed for colored drawings that uses a new outlier rejection algorithm for real-time tracking and surface deformation recovery. We present a content creation pipeline to efficiently create the 2-D and 3-D content. And, finally, we validate our work with two user studies that examine the quality of our texturing algorithm and the overall App experience.

A New 3D Augmented Reality Application for Educational Games to Help Children in Communication Interactively

“…We attempt to calculate the quaternion for simplifying the results. By applying the inertial tracking method as presented in [16] by Baldi et al, we then calculate and compute the quaternion, called r(t), using (4) where the value of α we used in this process is between 0 and 1. Representative threedimensional cartoons and augmented environments are then immediately overlaid onto the background we created to help and entertain children for educational games in a limited amount of time.…”

“…Their main aim is to help small children with autism spectrum condition only aged lower than 7 to be able to interact with the system. Moreover, Magnenat et al [4], proposed recently in 2015, from Disney Research Zurich developed a printed coloring book method for children. It is done by texturing and displaying the color characters in three dimensions using a smartphone in a children's coloring book.…”

StyleBlit: Fast Example‐Based Stylization with Local Guidance

“…The design of guiding channels is extremely important for achieving semantically meaningful transfer. The guidance can be relatively fuzzy with respect to a certain spatial location (e.g., segmentation or blurred gray‐scale gradients used by Hertzmann et al) or well‐localized and descriptive (e.g., a displacement field [SED16, FJS∗17], texture coordinates [RRFT14, MNZ∗15] or normal values [SMGG01, DBP∗15]). We call the latter local guidance.…”

Local pose estimation of feature points for object based augmented reality.

“…Several works explore the deformation of the object for visualization and content interaction (RENDL et al, 2014;TSE, 2008); folding of the object are also explored as interaction method (STEIMLE; JORDT; MAES, 2013). Changing the object appearance or state is also another way of interaction, such as painting a real image to change the virtual character appearance (MAGNENAT et al, 2015), or obtaining different simulations of the drawn objects (HAGBI et al, 2008).…”

“…On the other hand, different objects can have different behaviors when used by users. Not only restricted to rigid objects CHRISTENSEN, 2012) or planes (BILLINGHURST; POUPYREV, 2001) and its motion; but moreover, foldable TSE, 2008), deformable (STEIMLE; JORDT; MAES, 2013), or state changing (MAGNENAT et al, 2015) objects. Such objects can offer different kinds of interaction possibilities, such as illustrated in Figure 1.…”