Visual Attention-Aware Omnidirectional Video Streaming Using Optimal Tiles for Virtual Reality

Abstract

Owing to its interactive look around nature and very large resolution requirement, providing immersive omnidirectional video (ODV) streaming experiences in virtual reality applications demands cost-effective solutions to meet both the content delivery network and device constraints. In this paper, we introduce an adaptive ODV streaming pipeline that optimizes dynamic adaptive streaming over HTTP representations of ODV content considering their visual attention (VA) maps. The main contribution of this paper is the use of VA maps: 1) to compute a novel objective quality metric that captures the fact that not all of the ODV is actually watched by users: the VA spherical weightedbased objective quality measurement; 2) to define optimal tile representations of the ODV frames, namely, tiling schemes, which are composed of variable-sized and non-overlapping tiles; and 3) to efficiently distribute a given bitrate budget among the set of tiles within a tiling scheme for an ODV. We evaluate the proposed system performance with varying bandwidth conditions and the tracked head orientations from user experiments. Results show that the proposed system significantly outperforms the existing non-tiled and tile-based streaming solutions.

Do Users Behave Similarly in VR? Investigation of the User Influence on the System Design

“…In recent years, user-centric systems have been developed, optimising every step of the ODV video delivery chain: coding [30], streaming [36,44], caching [28,29], and rendering [40]. In particular, tile-based coding systems [30,34] were utilised using viewport adaptive streaming algorithms [8,33,38] to provide smooth VR video experience [36,44]. For instance, a navigationaware adaptation logic was developed in [44] to optimise the downloading rate for each tile of ODV streaming.…”

“…According to their results, their proposed algorithm can increase prediction accuracy of the expected viewport area by 13% on average compared to previous algorithms. By looking more at the server-side (i.e., coding optimisation), Ozcinar et al [36] proposed a visual attention-based ODV streaming system optimising the tile-based design taking into account users saliency maps. The work showed the importance of being user-centric also at encoding side without focusing a design of cost-aware VR system.…”

Subjective Evaluation of User Quality of Experience for Omnidirectional Video Streaming

“…Furthermore, Oznicar et al in [14] showed how the analysis of a user viewing angle can be beneficial in the process of network capacity planning since it is not necessary to transmit the segments of the ODV which are not in the current focus of the user. The same group of authors upgraded their method in [15].…”

Modern trends on quality of experience assessment and future work

“…In [125], the authors utilized random forest to predict the 360 attention, and then masked the non-content region. Differently, Yu et al [126] calculated PSNR in the attentive viewports only while Ozcinar et al [127] used ground-truth saliency maps directly. The F-PSNR can be directly applied to this saliency-based scheme as long as the saliency region is identified [12,115].…”