On Dependent Bit Allocation for Multiview Image Coding With Depth-Image-Based Rendering

Cheung, Gene; Velisavljević, Vladan; Ortega, Antonio

doi:10.1109/tip.2011.2158230

Cited by 78 publications

(54 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The disocclusion holes can be filled using inpainting algorithms [33], [34], but in general the larger the holes, higher the penalty in reconstruction quality. This monotonicity assumption is also demonstrated experimentally in [35]using a large number of multiview image sequences.…”

Section: B Free Viewpoint Video Modelmentioning

confidence: 59%

“…Furthermore, it should also monotonically increase with respect to the distance between the virtual view and the closer of the two captured views, and . Both of these tendencies are generally observed in empirical multiview data [7], [35]. If the virtual view is actually one of the anchor views, then the distortion should be zero.…”

Section: B Experiments and Simulationmentioning

confidence: 84%

See 1 more Smart Citation

Anchor View Allocation for Collaborative Free Viewpoint Video Streaming

Ren

Chan

Cheung

et al. 2015

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

Abstract-In free viewpoint video, a viewer can choose at will any camera angle or the so-called "virtual view" to observe a dynamic 3-D scene, enhancing his/her depth perception. The virtual view is synthesized using texture and depth videos of two anchor camera views via depth-image-based rendering (DIBR). We consider, for the first time, collaborative live streaming of a free viewpoint video, where a group of users may interactively pull and cooperatively share streams of different anchor views. There is a cost to access the anchor views from the live source, a cost to "reconfigure" the peer network due to a change in selected anchors during view switching, and a distortion cost due to the distance of the virtual views to the received anchor views at users. We optimize the anchor views allocated to users so as to minimize the overall streaming cost given by the access cost, reconfiguration cost, and view distortion cost. We first show that, if the reconfiguration cost due to view switching is negligible, the view allocation problem can be optimally and efficiently solved in polynomial time using dynamic programming. For the case of non-negligible reconfiguration cost, the problem becomes NP-hard. We thus present a locally optimal and centralized algorithm inspired by Lloyd's algorithm used in non-uniform scalar quantization. We further propose a distributed algorithm with convergence guarantee, where each peer group independently makes merge-and-split decisions with a well-defined fairness criteria. Simulation results show that our algorithms achieve low streaming cost due to its excellent anchor view allocation.

show abstract

Section: B Free Viewpoint Video Modelmentioning

confidence: 59%

Section: B Experiments and Simulationmentioning

confidence: 84%

Anchor View Allocation for Collaborative Free Viewpoint Video Streaming

Ren

Chan

Cheung

et al. 2015

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…The views have sub-sampled irregularly. In particular, we have selected 25 views, namely, 1,5,9,13,17,18,21,24,27,30,33,36,37,39,41,42,43,44,45,46,47,48,49,50,51, in the dataset of [36]. We have also tested our solution on a challenging test sequence, called new Tsukuba [39].…”

Section: B Applications To Existing Codersmentioning

confidence: 99%

“…The shortest path formulation permits to avoid a complex full search algorithm on both the number of reference views and their positions. Other works have proposed to optimally choose the set of views to be coded in a multi-view plus depth scenario [18], [19]. Among a set of views captured by the acquisition system, the sender is able to discard some views in the coding process and eventually synthesize them at the receiver.…”

Section: Introductionmentioning

confidence: 99%

Reference View Selection in DIBR-Based Multiview Coding

Maugey

Petrazzuoli

Frossard

et al. 2016

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Abstract-Augmented reality, interactive navigation in 3D scenes, multiview video and other emerging multimedia applications require large sets of images hence larger data volumes and increased resources compared to traditional video services. The significant increase of the number of images in multiview systems leads to new challenging problems in data representation and data transmission to provide high quality of experience on resource-constrained environments. In order to reduce the size of the data, different multi view video compression strategies have been proposed recently. Most of them use the concept of reference or key views that are used to estimate other images when there is high correlation in the dataset. In such coding schemes, the two following questions become fundamental: i) how many reference views have to be chosen for keeping a good reconstruction quality under coding cost constraints? ii) where to place these key views in the multiview dataset? As these questions are largely overlooked in the literature, we study the reference view selection problem and propose an algorithm for the optimal selection of reference views in multiview coding systems. Based on a novel metric that measures the similarity between the views, we formulate an optimization problem for the positioning of the reference views such that both the distortion of the view reconstruction and the coding rate cost are minimized. We solve this new problem with a shortest path algorithm that determines both the optimal number of reference views and their positions in the image set. We experimentally validate our solution in a practical multiview distributed coding system and in the standardized 3D-HEVC multi view coding scheme. We show that considering the 3D scene geometry in the reference view positioning problem brings significant rate-distortion improvements and outperforms traditional coding strategy that simply selects key frames based on the distance between cameras.Index Terms-Multiview distributed coding, key view positioning, inter-view correlation, view synthesis, multiview image coding

show abstract

“…For users who are observing the same free viewpoint video synchronized in time-e.g., during a live video broadcast of a public event like a piano recital-but not necessarily from the same viewpoint, they have incentive to pull texture and depth video streams from the same reference views, so that the streaming cost can be shared. On the other hand, it has been shown [3,4] that in general distortion of the synthesized view increases with its distance to the reference views. Thus, a user also has incentive to select video of reference views that tightly "sandwich" his chosen virtual view, in order to minimize visual distortion.…”

Section: Introductionmentioning

confidence: 99%

Optimizing peer grouping for live free viewpoint video streaming

Yuan

Cheung

et al. 2013

2013 IEEE International Conference on Image Processing

Self Cite

View full text Add to dashboard Cite

In free viewpoint video, a user can pull texture and depth videos captured from two nearby reference viewpoints to synthesize his chosen intermediate virtual view for observation via depth-image-based rendering (DIBR). For users who are observing the same video at the same time but not necessarily from the same virtual viewpoint, they have incentive to pull the same reference views so that the streaming cost can be shared. On the other hand, in general distortion of a synthesized virtual view increases with its distance to the reference views, and so a user also has incentive to select reference views that tightly "sandwich" his chosen virtual view, minimizing distortion. In a previous work, reference view sharing strategiesones that optimally trade off shared streaming costs with synthesized view distortions-were investigated for the case when users are first divided into groups, and each user group independently pulls two reference views and shares the resulting streaming cost. In this paper, we generalize the previous notion of user group, so that a user can simultaneously belong to two groups, and each group shares the streaming cost of a single view. We also aim to find a Nash Equilibrium (NE) solution of reference view selection, which is stable and from which no one has incentive to unilaterally deviate. Specifically, we first derive a lemma based on known properties of synthesized view distortion functions. We then design a search algorithm to find a NE solution, leveraging on the derived lemma to reduce search complexity. Experimental results show that the stable NE solution increases the overall cost only slightly when compared to the unstable optimal reference selection that gives the lowest overall cost. Further, a larger network will give a lower average cost for each user, and thus, users tend to join large networks for cooperation.

show abstract

On Dependent Bit Allocation for Multiview Image Coding With Depth-Image-Based Rendering

Cited by 78 publications

References 37 publications

Anchor View Allocation for Collaborative Free Viewpoint Video Streaming

Anchor View Allocation for Collaborative Free Viewpoint Video Streaming

Reference View Selection in DIBR-Based Multiview Coding

Optimizing peer grouping for live free viewpoint video streaming

Contact Info

Product

Resources

About