Real-Time Image Based Lighting for 360-Degree Panoramic Video

Abstract

The application of the newly popular content medium of 360 degree panoramic video to the widely used offline lighting technique of image based lighting is explored, and a system solution for real-time image based lighting of virtual objects using only the provided 360 degree video for lighting is developed. The system solution is suitable for use on live streaming video input, and is shown to run on consumer grade graphics hardware at the high resolutions and framerates necessary for comfortable viewing on head mounted displays, rendering at over 60 frames per second for stereo output at 1182x1464 per eye on a mid-range graphics card. Its use in several real-world applications is also studied, and extension to consider real-time shadowing and reflection is explored.

Distance Training of Higher Education Specialists Using Virtual Presence Technologies

“…This allows you to take a illustration of the event to a new level; actualization of the problem and requires a whole range of measures and the development of appropriate methodological material to create a realistic virtual environment that adequately simulates typical situations. It is advisable to apply approaches to creating virtual patients in medicine, a virtual laboratory base [3,4] with the technical focus of training.…”

“…There are models with a large number of standard lenses, for example: 6 (Insta 360 evo) ( Fig.1), 8 (VUZE 360) and even 24 (Surround × 24). The advantage of multi-lens models is a smaller number of geometric distortions compared to the image formed by wide-angle lenses, however, there is a greater likelihood of artifacts from combining images formed by different cameras and differences in illumination may occur [3][4][5]. In fact, to obtain panoramic video, the device registers video streams from different cameras, and then combines them -the so-called "stitching" (Fig.2).…”

A Novel Real-Time Virtual 3D Object Composition Method for 360° Video

“…These studies are based on VR, augmented reality, and mixed reality (MR). Focusing on research on 360 • video, Rhee et al implemented a real-time lighting and material expression of virtual objects, according to the positional change reconstructing the camera trajectory from the captured 360 • video [23]. Furthermore, the proposed MR360 is used to synthesize virtual objects with real background images.…”

“…Similarly, Tarko et al implemented real-time 3D composition using the Unity game engine through a stabilization process after camera tracking [25]. However, camera tracking was based on the captured image.…”

An Empirical Evaluation of the Performance of Real-Time Illumination Approaches: Realistic Scenes in Augmented Reality

“…Rhee et al [21] used a conventional low dynamic range (LDR) 360 • videos to render an interactive mixed reality scene. Iorns et al [22] developed a system to use a live streaming 360 • video as an input for image-based lighting where real-time shadowing and reflection were investigated. Michiels et al [23] used an appropriate 360 • environment map linked to a car position for real-time lighting added to the rendering equation.…”