Exploring Spherical Autoencoder for Spherical Video Content Processing

Zhou, Jin; Li, Na; Liu, Yao; Yao, Shuochao; Chen, Songqing

doi:10.1145/3503161.3548364

Cited by 2 publications

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Impressive research have been accomplished for coding [3,15,35,36], streaming [1,4,7,11,13,16,18,21,24,[29][30][31][32]37], FoV prediction [2,23,26], and edge-assisted delivery [6,19,27] of 360 video recently. However, a 360 video only covers the whole viewing sphere captured from a center position, and is typically projected onto a 2D plane and processed as a planar video.…”

Section: Related Workmentioning

confidence: 99%

Progressive Frame Patching for FoV-based Point Cloud Video Streaming

Zong¹,

Mao²,

Li³

et al. 2023

Preprint

View full text Add to dashboard Cite

Immersive multimedia applications, such as Virtual, Augmented and Mixed Reality, have become more and more practical and portable with the advances in hardware and software for acquiring and rendering 3D media as well as 5G/6G wireless networks. Many such applications require the delivery of volumetric video to users with six degrees of freedom (6-DoF) movements. Point Cloud has become a popular volumetric video format due to its flexibility and simplicity. A dense point cloud consumes much higher bandwidth than a 2D/360 degree video frame. User Field of View (FoV) is more dynamic with 6-DoF movement than 3-DoF movement. A user's view quality of a 3D object is affected by points occlusion and distance, which are constantly changing with user and object movements. To save bandwidth, FoV-adaptive streaming predicts a user's FoV and only downloads point cloud data falling in the predicted FoV. However, it is vulnerable to FoV prediction errors, which can be significant when a long buffer is utilized for smoothed streaming. In this work, we propose a multi-round progressive refinement framework for point cloud-based volumetric video streaming. Instead of sequentially downloading point cloud frames, our solution simultaneously downloads/patches multiple frames falling into a sliding time-window, leveraging on the inherent scalability of point-cloud coding. The optimal rate allocation among all tiles of active frames are solved analytically using the heterogeneous tile rate-quality functions calibrated by the predicted user FoV. Multi-frame downloading/patching simultaneously takes advantage of the streaming smoothness resulted from long buffer and the FoV prediction accuracy at short buffer length. We evaluate our streaming solution using simulations driven by real point cloud videos, real bandwidth traces, and 6-DoF FoV traces of real users. The experiments demonstrate that our solution is robust against the bandwidth/FoV prediction errors, and can delivery

show abstract