Light Source Estimation in Synthetic Images

Kasper, Mike; Keivan, Nima; Sibley, Gabe; Heckman, Christoffer

doi:10.1007/978-3-319-49409-8_72

Cited by 2 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Crucially, the illumination of an otherwise static scene may vary with time, or the materials in the scene may exhibit non-Lambertian reflectance. These effects are difficult to model analytically (although some attempts have been made in this context, e.g., [4], [8]) and require detailed knowledge of the geometry, illumination, and material properties of the scene.…”

Section: B Photometric (In)consistencymentioning

confidence: 99%

“…While some have attempted to circumvent this difficulty by treating visual localization as an end-to-end learning problem (e.g., [5], [6]), such end-to-end methods have yet to prove as accurate or robust as state-of-the-art methods based on well established geometric and probabilistic modelling [7]. On the other hand, analytical models of appearance must often make approximations or assumptions that are frequently violated in practice (e.g., photometric consistency), or require detailed knowledge of the geometry, illumination, and material properties of the environment [4], [8].…”

mentioning

confidence: 99%

See 1 more Smart Citation

How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change

Clement,

Kelly

2017

Preprint

View full text Add to dashboard Cite

Direct visual localization has recently enjoyed a resurgence in popularity with the increasing availability of cheap mobile computing power. The competitive accuracy and robustness of these algorithms compared to state-of-the-art featurebased methods, as well as their natural ability to yield dense maps, makes them an appealing choice for a variety of mobile robotics applications. However, direct methods remain brittle in the face of appearance change due to their underlying assumption of photometric consistency, which is commonly violated in practice. In this paper, we propose to mitigate this problem by training deep convolutional encoder-decoder models to transform images of a scene such that they correspond to a previouslyseen canonical appearance. We validate our method in multiple environments and illumination conditions using high-fidelity synthetic RGB-D datasets, and integrate the trained models into a direct visual localization pipeline, yielding improvements in visual odometry (VO) accuracy through time-varying illumination conditions, as well as improved metric relocalization performance under illumination change, where conventional methods normally fail. We further provide a preliminary investigation of transfer learning from synthetic to real environments in a localization context.

show abstract

Section: B Photometric (In)consistencymentioning

confidence: 99%

mentioning

confidence: 99%

How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change

Clement,

Kelly

2017

Preprint

View full text Add to dashboard Cite

show abstract

DeepLight: light source estimation for augmented reality using deep learning

Kán

Kafumann

2019

Vis Comput

View full text Add to dashboard Cite

This paper presents a novel method for illumination estimation from RGB-D images. The main focus of the proposed method is to enhance visual coherence in augmented reality applications by providing accurate and temporally coherent estimates of real illumination. For this purpose, we designed and trained a deep neural network which calculates a dominant light direction from a single RGB-D image. Additionally, we propose a novel method for real-time outlier detection to achieve temporally coherent estimates. Our method for light source estimation in augmented reality was evaluated on the set of real scenes. Our results demonstrate that the neural network can successfully estimate light sources even in scenes which were not seen by the network during training. Moreover, we compared our results with illumination estimates calculated by the state-of-the-art method for illumination estimation. Finally, we demonstrate the applicability of our method on numerous augmented reality scenes. Keywords Light source estimation • Augmented reality • Photometric registration • Deep learning (TUW). This research was funded by the Austrian research project WWTF ICT15-015. We thank Marc-Andé Gardner for providing us with the results of his algorithm through a Web service and for the kind explanations of details of the algorithm. We are also thankful to Alexander Pacha for advices about deep learning. We would like to thank NVIDIA Corporation for the donation of a Titan Xp graphics card and the Center for Geometry and Computational Design for access to a multi-GPU PC for training our neural networks. We also thank Min Kyung Lee, Iana Podkosova and Khrystyna Vasylevska for their support with controlled light experiments. Compliance with ethical standards Conflict of interest The authors declare that they have no conflict of interest. Ethical standard This research did not involve human participants.

show abstract

Light Source Estimation in Synthetic Images

Cited by 2 publications

References 12 publications

How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change

How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change

DeepLight: light source estimation for augmented reality using deep learning

Contact Info

Product

Resources

About