Joint video/depth/FEC rate allocation with considering 3D visual saliency for scalable 3D video streaming

Abstract

For robust video plus depth based 3D video streaming, video, depth and packet-level forward error correction (FEC) can provide many rate combinations with various 3D visual qualities to adapt to the dynamic channel conditions. Video/depth/FEC rate allocation under the channel bandwidth constraint is an important optimization problem for robust 3D video streaming. This paper proposes a joint video/depth/FEC rate allocation method by maximizing the receiver's 3D visual quality. Through predicting the perceptual 3D visual qualities of the different video/depth/FEC rate combinations, the optimal GOP-level video/depth/FEC rate combination can be found. Further, the selected FEC rates are unequally assigned to different levels of 3D saliency regions within each video/depth frame. The effectiveness of the proposed 3D saliency based joint video/depth/FEC rate allocation method for scalable 3D video streaming is validated by extensive experimental results.Index Terms-Video/depth rate allocation, FEC, 3D saliency, scalable 3D video streaming

Sliding-Window Forward Error Correction Based on Reference Order for Real-Time Video Streaming

“…The PFRs and PSNRs under different PLRs were experimentally analyzed using ( 1) the frame-level FEC scheme [7],…”

“…8, when the PLR is less than or equal to 6%, the three FEC schemes all exhibit high performance. When the PLR is 8%, 10%, 12%, 14%, 16%, 18%, and 20%, the PFR is respectively 0.939, 0.8841, 0.8535, 0.811, 0.7733, 0.7199, and 0.6823 for frame-level FEC [7] and 0.9729, 0.9421, 0.911, 0.8643, 0.8439, 0.791, and 0.7029 for time-order slidingwindow FEC [18]. When the proposed FEC scheme is used, the PFR is further increased to 0.992, 0.9891, 0.9886, 0.9825, 0.9783, 0.9493, and 0.8982, respectively.…”

“…In addition, the recovered packets of the current frame cannot help to recover the lost packets of previous frames, and the The associate editor coordinating the review of this manuscript and approving it for publication was Zilong Liu . distortion of the previous frames may propagate to the current and following frames. In GOP-level FEC, the FEC coding window contains all the video frames in a GOP to produce the corresponding FEC redundancies [7], [8]. By utilizing a large coding window, GOP-level FEC is capable of handling burst loss.…”

3D Video Rendering Adaptation: A Survey

“…Unlike the 2D video saliency, the 3D video saliency also deals with the depth saliency [30]. Hence, the 3D saliency is multi-dimensional conception which needs careful fusion computation [31].…”

Scalable 3D video streaming over P2P networks with playback length changeable chunk segmentation

“…For video plus depth based 3D video representation [5], Scalable Video Coding (SVC) [6] can be used to provide the flexible adaptations to the heterogeneous networking environments. To provide scalable virtual view qualities, SVC is sometimes independently used for video and depth [7] in the 3D video streaming.…”