Point Light Source Estimation based on Scenes Recorded by a RGB-D camera

Abstract

Estimation of the point light source position in the scene enhances the experience for augmented reality. The intensity image and depth information from the RGB-D camera allows estimation of the point light source position in a scene, where our approach does not need any probe objects or other measuring devices. The approach uses the Lambertian reflectance model, where the RGB-D camera provides the image and the surface normals and the remaining unknowns are the albedo and light parameters (light intensity and direction). In order to determine the light parameters, we assume that segments with a similar colour have the same albedo, which allows us to find the point light source that explains the illumination in the scene. The performance of this method is evaluated on multiple scenes, where a single light bulb is used to illuminate the scene. In this case, the average error in the angle between the true light position vector and our estimate is around 10 degrees. This allows realistic rendering of synthetic objects into the recorded scene, which is used to improve the experience of augmented reality.

Interactive light source position estimation for augmented reality with an RGB‐D camera

“…Other work focusses on using the light estimates for shape from shading to improve the depth maps given by the RGB‐D cameras . Our previous work focusses more on using the illuminant estimation in augmented reality, where computation time and compatibility with rendering software are more important than a perfect explanation of the underlying scene.…”

“…This paper is focussed towards the application of augmented reality, although the estimation of the illuminant is applicable in multiple domains (like surface improvement, scene understanding, illumination invariant scenes, etc). We have developed a methodology to estimate the location of a single point light source based on the image and the depth map provided by the Kinect, first described in . In this paper, we focus on making it applicable for augmented reality, by building a hybrid computer processing unit (CPU)–graphics processing unit (GPU) (i.e.…”

“…GPU implementation of methodology: We show that it is possible to run the method introduced in on a GPU, processing the incoming Kinect data with an average speed of more than 1 frame per second. This interactive implementation has potential benefits to graphics applications and game designers.…”

[POSTER] Retrieving Lights Positions Using Plane Segmentation with Diffuse Illumination Reinforced with Specular Component

“…Hara et al proposed in [3] to use an iterative scheme to retrieve the light source position but requires to know the geometry of the real scene. In [4] , Boom et al proposed a optimization method to retrieve the light position. However, it does not take into account the specular component and is limited to a single light point light retrieval.…”

Detecting Specular Reflections and Cast Shadows to Estimate Reflectance and Illumination of Dynamic Indoor Scenes

“…geometry, reflectance, illumination) as much as possible. In fact, constraining the MR user to have a single light source [2] or a textureless surface [3] reduces the range of possible scenarios. Also, even though approaches using light probes such as chrome spheres [4] or fisheye cameras [5] handle more generic scenes and deliver convincing augmentations, the use of such additional devices is neither practical nor adequate for wide-reaching MR systems.…”