Optimized Packet Scheduling in Multiview Video Navigation Systems

Abstract

International audienceWe study coding and transmission strategies in multi-camera systems, where correlated sources send data through a bottleneck channel to a central server, which eventually transmits views to different interactive users. We propose a dynamic navigation-path aware packet scheduling optimization under delay, bandwidth, and interactivity constraints aimed at optimizing the quality-of-experience of interactive users. In particular, the scene distortion is minimized jointly with the distortion variations along most likely navigation paths. The optimization relies both on a novel rate-distortion model, which captures the importance of each view in the scene reconstruction, and on an objective function that optimizes resources based on a client navigation model. The latter takes into account the distortion experienced by interactive clients as well as the distortion variations that might be observed by clients during multiview navigation. We solve the scheduling problem with a novel trellis-based solution, which permits to formally decompose the multivariate optimization problem thereby significantly reducing the computation complexity. Simulation results show the PSNR quality gain offered by the proposed algorithm compared to baseline scheduling policies. Finally, we show that the best scheduling policy consistently adapts to the most likely user navigation path and that it minimizes distortion variations that can be very disturbing for users in traditional navigation systems

Optimized Data Representation for Interactive Multiview Navigation

“…The increasing interactivity also brings challenges in data transmission, where the transmission policy needs to react to different requests of multiple users. In [7], [8], for example, the streaming of multiview video content in a navigation environment is studied, where the optimal transmission strategies are designed to provide high-quality content to heterogenous users under limited resources. All the above examples indicate that designing an interactive navigation system relates to many issues, including data representation, compression, transmission and rendering.…”

“…Indeed, the consideration of every component in isolation can only lead to suboptimal performance. In the literature, the individual components of the navigation system have been extensively R. Ma studied, e.g., 3-D scene representation [3], [4], multiview video compression [5], [6], multiview data streaming [7], [8] and view synthesis [9], [10]. However, there is clearly a lack of fully integrated frameworks that incorporate these techniques in an end-to-end system and jointly optimize them.…”

Optimal Representations for Adaptive Streaming in Interactive Multiview Video Systems

“…Streaming strategies and network optimization for multiview video, however, is still a relatively unexplored and new research topic. Optimal streaming strategies or reousrce allocation have been proposed in the case of resource-constrained networks [24]- [26], but not from a HAS systems perspecitve. A DASH-based stereo 3D video systems is introduced in [6], while a DASH-based multi-modal 3D video streaming system is proposed in [7].…”

Optimized receiver control in interactive multiview video streaming systems

“…We set the Markov transition matrix that allows transitions to adjacent states with probability 2p c /3 and two-state jumps with probability p c /3. Finally, to emulate heterogenous clients, we generate different navigation paths following the dynamic MV navigation model in [8] with minimum space between two adjacent views ∆ = 0.1. More specifically, we simulate i) a uniform navigation, when the user has the same probability of displaying the current view, or switching to the left or right view, and ii) a non-uniform navigation, when the user has a probability p n of displaying the current view and (1 − p n )/2 of switching to the left or right view.…”