Shader Lamps: Animating Real Objects With Image-Based Illumination

Abstract

We describe a new paradigm for three-dimensional computer graphics, using projectors to graphically animate physical objects in the real world. The idea is to replace a physical object-with its inherent color, texture, and material properties-with a neutral object and projected imagery, reproducing the original appearance directly on the object. Furthermore the projected imagery can be used to reproduce alternative appearances, including alternate shading, lighting, and even animation. Because the approach is to effectively "lift" the visual properties of the object into the projector, we call the projectors shader lamps .Some limited effects have previously been demonstrated along these lines for specific applications, however the real challenges to realizing this as a new medium for computer graphics lies in addressing the problems related to complete illumination of nontrivial physical objects. Our approach offers a very compelling method of visualization for a variety of applications including dynamic mechanical and architectural models, animated or "living" dioramas, artistic effects, entertainment, and even general visualization for problems that have meaningful physical shape representations. We present and demonstrate methods for using multiple shader lamps to animate physical objects of varying complexity, from a flower vase, to some wooden blocks, to a model of the Taj Mahal.

The personal cockpit

“…Spatial augmented reality (SAR), originally demonstrated by Raskar et al [173][174][175], uses projectors to overlay virtual content on real-world surfaces. This concept is similar to the use of transformed projection in CAVE displays; however, projections are mapped to the underlying structure of the given projection surface [177,209].…”

Adaptive stray-light compensation in dynamic multi-projection mapping

“…The impact of scattered light from projection mapping was first discussed by Raskar et al [4]. They argued that scattered light contributes to the realism of the projection as it generates global illumination effects.…”

Intraoperative Visual Guidance and Control Interface for Augmented Reality Robotic Surgery

“…Direct Augmentations is also called spatial augmented reality (SAR) using projector-camera system to present the virtual information directly in the 3D real world environment (Raskar et al, 2001;Bimber & Raskar, 2004). Based on the above different approaches of augmentations, four kinds of AR platform are popularly used in present surgical simulation and treatment: head-mounted display (HMD) based AR (Cakmakci & Rolland, 2006), semi-transparent mirror based AR (Liao et al, 2010) and projector based AR (Bimber & Raskar, 2004).…”