Consistent interactive augmentation of live camera images with correct near-field illumination

Grosch, Thorsten; Eble, Tobias; Müeller, Stefan

doi:10.1145/1315184.1315207

Cited by 40 publications

(21 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A method for real-time global illumination of indoor scenes with diffuse materials, lit by environment lighting was proposed in [GEM07]. The direct light from outside is used to update a near-field representation of the indirect light in the room by a dynamic form of the irradiance volume.…”

Section: Interactive Differential Renderingmentioning

confidence: 99%

Photorealistic rendering of mixed reality scenes

Kronander

Banterle

Gardner

et al. 2015

Computer Graphics Forum

View full text Add to dashboard Cite

Photo-realistic rendering of virtual objects into real scenes is one of the most important research problems in computer graphics. Methods for capture and rendering of mixed reality scenes are driven by a large number of applications, ranging from augmented reality to visual effects and product visualization. Recent developments in computer graphics, computer vision, and imaging technology have enabled a wide range of new mixed reality techniques including methods for advanced image based lighting, capturing spatially varying lighting conditions, and algorithms for seamlessly rendering virtual objects directly into photographs without explicit measurements of the scene lighting. This report gives an overview of the state-of-the-art in this field, and presents a categorization and comparison of current methods. Our in-depth survey provides a tool for understanding the advantages and disadvantages of each method, and gives an overview of which technique is best suited to a specific problem.

show abstract

Section: Interactive Differential Renderingmentioning

confidence: 99%

Photorealistic rendering of mixed reality scenes

Kronander

Banterle

Gardner

et al. 2015

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…In Korn et al [34], two such HDR video cameras were used to estimate the distance of moving light sources. In Grosch et al [19], a near-field illumination approach was presented for augmentations under daylight in a real room. The idea of differential rendering has been applied to several interactive global illumination methods, including instant radiosity [32,33], progressive path tracing [28] and light propagation volumes [13].…”

Section: Previous Workmentioning

confidence: 99%

Interactive near-field illumination for photorealistic augmented reality on mobile devices

Rohmer

Büschel²,

Dachselt³

et al. 2014

2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Self Cite

View full text Add to dashboard Cite

Mobile devices become more and more important today, especially for augmented reality (AR) applications in which the camera of the mobile device acts like a window into the mixed reality world. Up to now, no photorealistic augmentation is possible since the computational power of the mobile devices is still too weak. Even a streaming solution from a stationary PC would cause a latency that affects user interactions considerably. Therefore, we introduce a differential illumination method that allows for a consistent illumination of the inserted virtual objects on mobile devices, avoiding a delay. The necessary computation effort is shared between a stationary PC and the mobile devices to make use of the capacities available on both sides. The method is designed such that only a minimum amount of data has to be transferred asynchronously between the stationary PC and one or multiple mobile devices. This allows for an interactive illumination of virtual objects with a consistent appearance under both temporally and spatially varying real illumination conditions. To describe the complex near-field illumination in an indoor scenario, multiple HDR video cameras are used to capture the illumination from multiple directions. In this way, sources of illumination can be considered that are not directly visible to the mobile device because of occlusions and the limited field of view of built-in cameras.

show abstract

“…Furthermore, several global illumination approaches for augmented reality scenarios exist [19][20] [21], which perform pre-calculations based on a static real scene to achieve interactive framerates. For those approaches only the virtual content is allowed to be dynamic.…”

Section: Related Workmentioning

confidence: 99%

Instant indirect illumination for dynamic mixed reality scenes

Lensing

Broll

2012

2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

View full text Add to dashboard Cite

For seamless integration of virtual content into real scenes, realizing mutual global lighting effects between both worlds belongs to the most important and challenging goals. Therefore, plenty of global illumination approaches exist, which mostly share the same restriction: the real scene is approximated by a static model, which was built in advance and thus has to remain static. In our paper, we propose an image-space global illumination approach, based on reflective shadow maps, combined with the use of an RGB-D camera, to simulate first bounce diffuse indirect illumination without any pre-computations. Our approach supports indirect illumination in both directions (real to virtual and vice versa) and runs in real-time. Furthermore, it does not require advanced shader properties, since we developed an implementation making efficient usage of the Z-Buffer algorithm for calculating indirect illumination. INTRODUCTIONDue to the fast development of modern graphic cards, real-time global illumination (GI) became a vast area of research. Indirect illumination effects, which were previously seen in offline rendering only, become more and more applicable to real-time applications. Since many real-time GI approaches require additional calculations on the scene in advance, they typically restrict dynamic changes to rigid body motion or approximate indirect illumination for low frequency behavior only. Both approaches are not suitable for mixed reality applications, where the captured real scene typically introduces all kinds of dynamic movements to be instantly considered. This is somehow annoying for mixed reality applications, as due to the blending of virtual and real content, global illumination effects become more important to provide seamless integration of virtual content (see Figure 1). Nevertheless, global illumination approaches exist, which are real-time capable and also account for high frequency indirect light behavior. Dachsbacher's Reflective Shadow Maps (RSM) [1] represent such an approach. As it is robust, it provides a foundation for several recent GI algorithms [2][3]. The performance of indirect lighting with RSM can be much improved if the lighting calculation is restricted to the visible part of the scene only [4], which can be achieved by screen-space deferred lighting. This restriction to screen-space calculations allows for new opportunities in the area of mixed reality. Employing a RGB-D camera allows us to obtain a coarse spatial representation of the visible scene which then may be used for GI calculations. For the refinement of the coarse spatial representation we propose guided image filtering [5] where the RGB-Image serves as guidance image to improve the depth image. During our research we observed that indirect illumination with RSM provides acceptable results to mixed reality applications, if the number of virtual point lights (VPL) introducing indirect light based on the RSM, is very high (4-8k). For such high VPL counts the fillrate becomes the bottleneck of an implementation. Thus, o...

show abstract

Consistent interactive augmentation of live camera images with correct near-field illumination

Cited by 40 publications

References 43 publications

Photorealistic rendering of mixed reality scenes

Photorealistic rendering of mixed reality scenes

Interactive near-field illumination for photorealistic augmented reality on mobile devices

Instant indirect illumination for dynamic mixed reality scenes

Contact Info

Product

Resources

About