RaViS: Real-time accelerated View Synthesizer for immersive video 6DoF VR

Bonatto, Daniele; Fachada, Sarah; Lafruit, Gauthier

doi:10.2352/issn.2470-1173.2020.13.ervr-382

Cited by 17 publications

(18 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This pipeline is shown in Figure 4. The warping and blending operations are performed alternatively for each input image using OpenGL [32] or on the CPU [15].…”

Section: Rvs In Practicementioning

confidence: 99%

View Synthesis Tool for VR Immersive Video

Fachada¹,

Bonatto²,

Teratani³

et al. 2022

Computer Game Development

Self Cite

View full text Add to dashboard Cite

This chapter addresses the view synthesis of natural scenes in virtual reality (VR) using depth image-based rendering (DIBR). This method reaches photorealistic results as it directly warps photos to obtain the output, avoiding the need to photograph every possible viewpoint or to make a 3D reconstruction of a scene followed by a ray-tracing rendering. An overview of the DIBR approach and frequently encountered challenges (disocclusion and ghosting artifacts, multi-view blending, handling of non-Lambertian objects) are described. Such technology finds applications in VR immersive displays and holography. Finally, a comprehensive manual of the Reference View Synthesis software (RVS), an open-source tool tested on open datasets and recognized by the MPEG-I standardization activities (where”I″ refers to”immersive”) is described for hands-on practicing.

show abstract

“…This pipeline is shown in Figure 4. The warping and blending operations are performed alternatively for each input image using OpenGL [32] or on the CPU [15].…”

Section: Rvs In Practicementioning

confidence: 99%

View Synthesis Tool for VR Immersive Video

Fachada¹,

Bonatto²,

Teratani³

et al. 2022

Computer Game Development

Self Cite

View full text Add to dashboard Cite

show abstract

“…Though only two virtual views following the user's pose must be rendered at any time, the high frame rates used in VR (60-120 fps) impose stringent real-time constraints. Optimizing an in-house OpenGL implementation of RVS -called RaViSwe reach real-time performances, generating two synthesized views at 60 to 90 fps [2].…”

Section: View Synthesis In Stereoscopic Hmdmentioning

confidence: 99%

“…This paper presents a demonstration centered on the topic of view synthesis that is complementary to the tutorial session "The MPEG Immersive Video coding standard" to be held at the Visual Communication and Image Processing (VCIP) conference 2021. Our approach [1], [2], we will demonstrate at VCIP for Light-Field VR, is based on Depth Image Based Rendering (DIBR): the pixels from input views are first projected to 3D space using their depth maps, and afterwards reprojected to any virtual view one wishes to synthesize.…”

Section: Introductionmentioning

confidence: 99%

MPEG Immersive Video tools for Light Field Head Mounted Displays

Bonatto

Hirt²,

Kvasov³

et al. 2021

2021 International Conference on Visual Communications and Image Processing (VCIP)

Self Cite

View full text Add to dashboard Cite

Light-Field displays project hundreds of microparallax views for users to perceive 3D without wearing glasses. It results in gigantic bandwidth requirements if all views would be transmitted, even using conventional video compression per view. MPEG Immersive Video (MIV) follows a smarter strategy by transmitting only key images and some metadata to synthesize all the missing views. We developed (and will demonstrate) a realtime Depth Image Based Rendering software that follows this approach for synthesizing all Light-Field micro-parallax views from a couple of RGBD input views.

show abstract

“…However, for certain content types (especially natural scenes with high levels of motion) it still produces noticeable artifacts (e.g. bad blending and disocclusions) that often degrade the quality of experience [5]. Efforts to improve the performance of TMIV have primarily focused on improving the depth estimation and on the synthesis at the decoder side [6,7].…”

Section: Introductionmentioning

confidence: 99%

A CNN-based Post-Processor for Perceptually-Optimized Immersive Media Compression

Katsenou¹,

Zhang²,

Bull³

2022

Preprint

View full text Add to dashboard Cite

In recent years, resolution adaptation based on deep neural networks has enabled significant performance gains for conventional (2D) video codecs. This paper investigates the effectiveness of spatial resolution resampling in the context of immersive content. The proposed approach reduces the spatial resolution of input multi-view videos before encoding, and reconstructs their original resolution after decoding. During the up-sampling process, an advanced CNN model is used to reduce potential re-sampling, compression and synthesis artefacts. This work has been fully tested with the TMIV coding standard using a Versatile Video Coding (VVC) codec. The results demonstrate that the proposed method achieves significant rate quality performance improvement for the majority of the test sequences, with an average BD-VMAF improvement of 3.07 over all sequences.

show abstract

RaViS: Real-time accelerated View Synthesizer for immersive video 6DoF VR

Abstract: Fast track article for IS&T International Symposium on Electronic Imaging 2020: The Engineering Reality of Virtual Reality proceedings.

Cited by 17 publications

References 17 publications

View Synthesis Tool for VR Immersive Video

View Synthesis Tool for VR Immersive Video

MPEG Immersive Video tools for Light Field Head Mounted Displays

A CNN-based Post-Processor for Perceptually-Optimized Immersive Media Compression

Contact Info

Product

Resources

About