Interactive near-field illumination for photorealistic augmented reality on mobile devices

Abstract

Mobile devices become more and more important today, especially for augmented reality (AR) applications in which the camera of the mobile device acts like a window into the mixed reality world. Up to now, no photorealistic augmentation is possible since the computational power of the mobile devices is still too weak. Even a streaming solution from a stationary PC would cause a latency that affects user interactions considerably. Therefore, we introduce a differential illumination method that allows for a consistent illumination of the inserted virtual objects on mobile devices, avoiding a delay. The necessary computation effort is shared between a stationary PC and the mobile devices to make use of the capacities available on both sides. The method is designed such that only a minimum amount of data has to be transferred asynchronously between the stationary PC and one or multiple mobile devices. This allows for an interactive illumination of virtual objects with a consistent appearance under both temporally and spatially varying real illumination conditions. To describe the complex near-field illumination in an indoor scenario, multiple HDR video cameras are used to capture the illumination from multiple directions. In this way, sources of illumination can be considered that are not directly visible to the mobile device because of occlusions and the limited field of view of built-in cameras.

Estimating Illumination Direction for Augmented Reality in Real-Time by using Low-Cost Sensors

“…Other approaches use special hardware to generate a 3d reconstruction of the scene ( [16,17]). Thus, by knowing the location of the objects and the light sources, they can estimate the illumination of the virtual objects.…”

“…Thus, by knowing the location of the objects and the light sources, they can estimate the illumination of the virtual objects. Rohmer et al ( [16]) presented a differential illumination method obtaining a consistent illumination of the inserted virtual objects on mobile devices. They use multiple HDR video cameras in a predetermined scenario.…”

“…As Rohmer et al proposed in [16], the computation of the system is distributed between the smartphone application and a PC application. Hence, the calculation of the direction of the light is done in the PC, allowing the smartphone application to focus only on the processing of the AR application.…”

GPS-Based Augmented Reality System of Past Topography Using Antique Maps

“…Also, if it is necessary for an administrative reason, as for the ancient/old books, the maps of photoprint-versions or traditional cartography can also be accounted as the antique maps [6][7][8][9][10][11][12][13][14][15]. In this paper, we define Korean antique maps as the ones that had been printed or copied with the traditional Korean production technique prior to 1910, excluding modern surveying technology.…”

Detecting Specular Reflections and Cast Shadows to Estimate Reflectance and Illumination of Dynamic Indoor Scenes

“…In fact, constraining the MR user to have a single light source [2] or a textureless surface [3] reduces the range of possible scenarios. Also, even though approaches using light probes such as chrome spheres [4] or fisheye cameras [5] handle more generic scenes and deliver convincing augmentations, the use of such additional devices is neither practical nor adequate for wide-reaching MR systems. Consequently, several state-of-the-art methods aim to use a single RGB or RGB-D camera for photometric registration.…”