Realtime Estimation of Illumination Direction for Augmented Reality on Mobile Devices

This paper presents the augmentation of immersive omnidirectional video with realistically lit objects. Recent years have known a proliferation of real-time capturing and rendering methods of omnidirectional video. Together with these technologies, rendering devices such as Oculus Rift have increased the immersive experience of users. We demonstrate the use of structure from motion on omnidirectional video to reconstruct the trajectory of the camera. The position of the car is then linked to an appropriate 360 • environment map. State-of-the-art augmented reality applications have often lacked realistic appearance and lighting. Our system is capable of evaluating the rendering equation in real-time, by using the captured omnidirectional video as a lighting environment. We demonstrate an application in which a computer generated vehicle can be controlled through an urban environment.

show abstract

Realtime Estimation of Illumination Direction for Augmented Reality on Mobile Devices

Cited by 22 publications

References 17 publications

LightNet: Deep Learning Based Illumination Estimation from Virtual Images

LightNet: Deep Learning Based Illumination Estimation from Virtual Images

A Comparative Survey of Geometric Light Source Calibration Methods

Interactive Augmented Omnidirectional Video with Realistic Lighting

Contact Info

Product

Resources

About