Viewport-aware adaptive 360° video streaming using tiles for virtual reality

Abstract

360°video is attracting an increasing amount of attention in the context of Virtual Reality (VR). Owing to its very high-resolution requirements, existing professional streaming services for 360°video suffer from severe drawbacks. This paper introduces a novel end-to-end streaming system from encoding to displaying, to transmit 8K resolution 360°v ideo and to provide an enhanced VR experience using Head Mounted Displays (HMDs). The main contributions of the proposed system are about tiling, integration of the MPEG-Dynamic Adaptive Streaming over HTTP (DASH) standard, and viewport-aware bitrate level selection. Tiling and adaptive streaming enable the proposed system to deliver very high-resolution 360°video at good visual quality. Further, the proposed viewport-aware bitrate assignment selects an optimum DASH representation for each tile in a viewportaware manner. The quality performance of the proposed system is verified in simulations with varying network bandwidth using realistic view trajectories recorded from user experiments. Our results show that the proposed streaming system compares favorably compared to existing methods in terms of PSNR and SSIM inside the viewport.

COCO-GAN: Generation by Parts via Conditional Coordinating

“…It is hard to generate full-scene for VR in real time using standard generative models. Some recent VR studies on omnidirectional view rendering and streaming [6,25,5] focus on reducing computational cost or network bandwidth by adapting to the user's viewport. COCO-GAN, with the generation-by-parts feature, can easily inherit the same strategy and achieve computation on-demand with respect to the user's viewpoint.…”

Network Support for AR/VR and Immersive Video Application: A Survey

“…They describe a configurable implementation of these technologies, within the GPAC open-source player, allowing experimentations of different adaptation policies for tiled video content. (Ozcinar et al, 2017b) introduced a novel endto-end streaming system from encoding to displaying, to transmit 8K resolution 360 degree video and to provide an enhanced VR experience using Head Mounted Displays (HMDs). The main contributions of the proposed system are about tiling, integration of the MPEGDynamic Adaptive Streaming over HTTP (DASH) standard, and viewport-aware bitrate level selection.…”

Virtual Transcendence Experiences

“…The majority of today's VR content representation and processing efforts are tiles-based and exploit adaptive streaming paradigm. Other options are view-port aware bitrate adaptation [7], MPEG's Dynamic Adaptive Streaming over HTTP (DASH) [8] and a tile-based processing using rectangular tiles for high-resolution media [9]. Another option could be a nontile based but viewport-adaptive, 360° video delivery system by creating spherical scene representations with various Quality Emphasis Regions (QER) [10].…”