View Synthesis Tool for VR Immersive Video

Fachada, Sarah; Bonatto, Daniele; Teratani, Mehrdad; Lafruit, Gauthier

doi:10.5772/intechopen.102382

Cited by 7 publications

(13 citation statements)

References 38 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Reference View Synthesis Software (RVS) [2,11,13,12] is an open-source DIBR view synthesizer of the MPEG Immersive Video group for their new standard: MIV [16]. Using only four 1080p input views and their corresponding depth maps, RVS synthesizes novel views up to 2 × 90 frames per second (fps) in a head mounted display (HMD) using common hardware (NVIDIA GTX 1080Ti) while outperforming the old reference software VSRS [17] by an average of 2.5 dB PSNR [18].…”

Section: Related Workmentioning

confidence: 99%

“…Using a ping-pong buffer, the synthesized views at the final pose are blended together. To avoid classical view synthesis issues [11], a weight is given to discard unfit triangles.…”

Section: Related Workmentioning

confidence: 99%

“…To synthesize novel views on embedded devices, the DIBR method is selected for its advantages [11], however, RaViS uses the OpenGL API, not available on such devices. This leaves WebGL, OpenGL ES or Vulkan as a replacement for it.…”

Section: Proposed Approachmentioning

confidence: 99%

“…Potentially, by using a low number of input views, a high quality natural scene synthesis can be reached in real-time. This is case of the Reference View Synthesis Software (RVS) [2,11,12,13] which is a Depth Image-Based Rendering (DIBR) method as for each reference view, a corresponding depth map is provided.…”

Section: Introductionmentioning

confidence: 99%

“…This work demonstrates a port of RaViS [13], a state-of-theart GPU accelerated DIBR method based on the Reference View Synthesis (RVS) [2,11,13,12] to target embedded devices, and in particular an embedded Head Mounted Display (HMD).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Novel view synthesis in embedded virtual reality devices

Laurie¹,

Daniele²,

Sarah³

et al. 2022

Self Cite

View full text Add to dashboard Cite

Virtual Reality and Free Viewpoint navigation require highquality rendered images to be realistic. Current hardware assisted raytracing methods cannot reach the expected quality in real-time and are also limited by the 3D mesh quality. An alternative is depth image-based rendering (DIBR) where the input only consists of images and their associated depth maps for synthesizing virtual views to the Head Mounted Display (HMD). The MPEG Immersive Video (MIV) standard uses such a DIBR algorithm called the Reference View Synthesizer (RVS). We have first implemented a GPU version, called the Real-time Accelerated View Synthesizer (RaViS), that synthesizes two virtual views in real-time for the HMD. In the present paper, we explore the differences between desktop and embedded GPU platforms, porting RaViS to an embedded HMD without the need for a separate, discrete desktop GPU. The proposed solution gives a first insight into DIBR View Synthesis techniques in embedded HMDs using OpenGL and Vulkan, a cross-platform 3D rendering library with support for embedded devices.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Using a ping-pong buffer, the synthesized views at the final pose are blended together. To avoid classical view synthesis issues [11], a weight is given to discard unfit triangles.…”

Section: Related Workmentioning

confidence: 99%

Section: Proposed Approachmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Novel view synthesis in embedded virtual reality devices

Laurie¹,

Daniele²,

Sarah³

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

Toward physically realistic vision in teleoperation: A user study with light‐field head mounted display and 6‐DoF head motion

Bechtel,

Weber,

Severin

et al. 2023

J Soc Info Display

View full text Add to dashboard Cite

Besides haptics, the visual channel provides the most essential feedback to the operator in teleoperation setups. For optimal performance, the view on the remote scene must provide 3D information, be sharp, and of high resolution. Head‐mounted displays (HMD) are applied to improve the immersion of the operator into the remote environment. Still, so far, no near‐eye display technology was available that provides a natural view on objects within the typical manipulation distance (up to 1.2 m). The main limitation is a mismatch of the 3D distance and the focal distance of the visualized objects (vergence‐accommodation conflict) in displays with fixed focal distance. This conflict potentially leads to eye strain after extended use. Here, we apply a light‐field HMD providing close‐to‐continuous depth information to the user, thus avoiding the vergence‐accommodation conflict. Furthermore, we apply a time‐of‐flight sensor to generate a 2.5D environment model. The displayed content is processed with image‐based rendering allowing a 6 degree‐of‐freedom head motion in the visualized scene. The main objective of the presented study is evaluating the effects of view perspective and light‐field on performance and workload in a teleoperation setup. The reduction of visual effort for the user is confirmed in an abstract depth‐matching task.

show abstract

Pattern-free Plenoptic 2.0 Camera Calibration

Fachada

Bonatto

Losfeld

et al. 2022

2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)

Self Cite

View full text Add to dashboard Cite

The plenoptic 2.0 camera is a light field acquisition system consisting of a main lens and a micro-lens array (MLA) at a non-focal distance of the main lens. While it allows to retrieve the geometry of the scene, the distances between the main lens, the MLA and the sensor are usually unknown. Therefore, the use cases for plenoptic cameras stay limited while they have more potential applications such as virtual reality, provided that their camera parameters are precisely known. In this paper, we present a pattern-free calibration method to retrieve the plenoptic camera's intrinsic parameters from the rendered subaperture images, i.e. the distance from main lens to the MLA and sensor, the focal length and the principal point. The proposed method utilises the relation between scene's distances, i.e. depth, and the subaperture images' micro-disparities, which contains information about the camera parameters. To the best of our knowledge, it is the first pattern-free calibration method for plenoptic 2.0 cameras. We compare the parameters obtained using our method applied to subaperture images rendered from three different software tools (Plenoptic Toolbox, Reference Lenslet content Convertor, and Lenslet to Multiview) with an open-source pattern-based method (Compote). The proposed pattern-free calibration method has consistent camera parameters with the traditional pattern-based calibration method. We therefore reliably obtain the intrinsic parameters of any plenoptic camera from its captured images, even in the absence of any calibration pattern.

show abstract

View Synthesis Tool for VR Immersive Video

Cited by 7 publications

References 38 publications

Novel view synthesis in embedded virtual reality devices

Novel view synthesis in embedded virtual reality devices

Toward physically realistic vision in teleoperation: A user study with light‐field head mounted display and 6‐DoF head motion

Pattern-free Plenoptic 2.0 Camera Calibration

Contact Info

Product

Resources

About