Real-time estimation of light source environment for photorealistic augmented reality

Abstract

This paper proposes a vision-based augmented reality system with correct representation of attached and cast shadows. To realize a seamless augmented reality system, we need to resolve a number of problems. Especially, the geometric and photometric registration problems are important. These problems require to estimate the positions of light sources and user's viewpoint. The proposed system resolves the problems using a 3D marker which combines a 2D square marker and a mirror ball. The 2D marker and the mirror ball are used to estimate the relationship between the real and virtual worlds and the positions of light sources in the real world, respectively.

Shadow Generation in Mixed Reality: A Comprehensive Survey

“…Haller et al [19] generated hard volume shadows on other objects using the phantom technique, which will be explained in geometrically based hard shadow generation. Kanbara et al [20] employed shadows in AR focusing on light source detection, which forms the basis of work done by [3]. Jacobs et al [21] focused on colour-consistent virtual shadows in a real scene.…”

Probeless Illumination Estimation for Outdoor Augmented Reality

“…The light probe image must be acquired in High Dynamic Range (HDR) to capture illumination sources without pixel saturation. This approach was pioneered in (Debevec, 1998) and has later been employed as the basic illumination technique in a multitude of real-time AR research (Barsi et al, 2005); (Franke & Jung, 2008); (Havran et al, 2005); (Kanbara & Yokoya, 2004); (Madsen & Laursen, 2007). While the approach of using a captured HDR environment map of the scene allows for really realistic illumination interaction between real and virtual scene elements, it is of course totally impractical for an adaptive real-time system to require such environment maps.…”

Outdoor Illumination Estimation in Image Sequences for Augmented Reality

“…The drawbacks are that it is time-consuming and impractical to acquire the environment map whenever something has changed in the scene, for example the illumination. Illumination adaptive techniques based on the environment map idea have been demonstrated in (Havran et al, 2005;Kanbara and Yokoya, 2004) but require a prototype omni-directional HDR camera, or a reflective sphere placed in the scene, respectively.…”