SHVC Tile-Based 360-Degree Video Streaming for Mobile VR: PC Offloading Over mmWave

Abstract

360-degree video streaming for high-quality virtual reality (VR) is challenging for current wireless systems because of the huge bandwidth it requires. However, millimeter wave (mmWave) communications in the 60 GHz band has gained considerable interest from the industry and academia because it promises gigabit wireless connectivity in the huge unlicensed bandwidth (i.e., up to 7 GHz). This massive unlicensed bandwidth offers great potential for addressing the demand for 360-degree video streaming. This paper investigates the problem of 360-degree video streaming for mobile VR using the SHVC, the scalable of High-Efficiency Video Coding (HEVC) standard and PC offloading over 60 GHz networks. We present a conceptual architecture based on advanced tiled-SHVC and mmWave communications. This architecture comprises two main parts. (1) Tile-based SHVC for 360-degree video streaming and optimizing parallel decoding. (2) Personal Computer (PC) offloading mechanism for transmitting uncompressed video (viewport only). The experimental results show that our tiled extractor method reduces the bandwidth required for 360-degree video streaming by more than 47% and the tile partitioning mechanism was improved by up to 25% in terms of the decoding time. The PC offloading mechanism was also successful in offloading 360-degree decoded (or viewport only) video to mobile devices using mmWave communication and the proposed transmission schemes.

An Edge and Fog Computing Platform for Effective Deployment of 360 Video Applications

“…Authors in [7] advocate the adoption of mmWave communications along with mobile edge computing and proactive caching mechanisms in order to meet strict latency and reliability requirements, while multi-path multi-tier 360 video streaming solutions for 5G wireless networks are designed and evaluated in [8], by analyzing the optimal trade-off between video quality enhancement and cost for using the 5G channel. In [9], the Scalable extension of the High-Efficiency Video Coding (HEVC) standard is employed to develop a Tile-Based video streaming solution over mmWave networks together with an offloading mechanism to assist mobile terminals in decoding the viewport. On the other hand, the idea of predicting users head movements in order for the user to download only the FoV is discussed in [10], yet prediction errors are inevitably introduced.…”

Region of interest–based segmented tiled adaptive streaming using head-mounted display tracking sensing data

“…The coordinates of the vertexes of HMD output area can be calculated using the seeing direction (f, u, c) and the values of U, V after applying equations (10) to (8)…”

“…To render on display devices such as HMDs, the pixel coordinate (X e , Y e ) corresponding to that of render target texture in the entire 360 VR video is obtained by applying equation (9). By using the calculated coordinates, equation (8) extracts the corresponding output range using equation (12). In equation (12), w is the horizontal size of display output device and k is its vertical size…”

“…6 That is, it causes a prediction mismatch, and the solution, which creates the generated reference picture, still has overhead problems. 6,8–10 Many studies have been conducted with the objective of solving the problem outlined above. 4 However, it requires H/W decoding to provide high-quality 360 VR streaming with 8K and 16K resolution video.…”

“…4,6,7 Furthermore, ordinary horizontal FOV is defined as 120°, which demands a 360 VR viewport, and the FOV corresponding to the VR viewport requires 4K resolution. Therefore, a minimum of 12K resolution and 100 FPS is necessary to stream high-quality 360 VR videos and reduce VR sickness, 8 which is another challenge.…”