Reference frame selection for loss-resilient texture &amp;amp; depth map coding in multiview video conferencing

Macchiavello, Bruno; Dorea, Camilo; Hung, Edson M.; Cheung, Gene; Tan, Wai-Tian

doi:10.1109/icip.2012.6467194

Cited by 12 publications

(22 citation statements)

References 13 publications

(12 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The works in [14] propose to exploit the flexibility provided by reference picture selection (RPS) in H.264 video coding standard for real-time encoded depth video, so that a depth block important to the quality of the synthesized view can be predicted from a transmitted frame further in the past for more reliable decoding [15]. The authors in [16], [17] propose to use distributed source coding (DSC) [18] to enable both periodic view-switching in multiview video and loss resiliency for peers watching the same multiview video synchronized in time but not in view.…”

Section: A Multiview Video Streaming and High-dimensional Media Navimentioning

confidence: 99%

“…For simplicity, we consider the following distortion function , which satisfies the two assumptions of monotonicity in Section III-B: (14) where is the weighting parameter for distortion in the total cost. The rate at which the distortion increases with the distance between anchor views can be adjusted by the parameters and .…”

Section: B Experiments and Simulationmentioning

confidence: 99%

“…The results of those simulations are qualitatively the same as what is presented here, and hence are not shown for brevity). The distortion of a peer is calculated according to Equation (14), and the total cost of all peers is calculated according to Equation (7).…”

Section: B Experiments and Simulationmentioning

confidence: 99%

“…6 shows the average PSNR observed by the peers as a function of the price of camera views. The peer PSNR is calculated using the MSE distortion model in Equation (14). The View-independent approach has optimal PSNR since the peers always pull anchor views to minimize their own distortion without considering access cost or network reconfiguration.…”

Section: B Experiments and Simulationmentioning

confidence: 99%

See 3 more Smart Citations

Anchor View Allocation for Collaborative Free Viewpoint Video Streaming

Ren

Chan

Cheung

et al. 2015

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

Abstract-In free viewpoint video, a viewer can choose at will any camera angle or the so-called "virtual view" to observe a dynamic 3-D scene, enhancing his/her depth perception. The virtual view is synthesized using texture and depth videos of two anchor camera views via depth-image-based rendering (DIBR). We consider, for the first time, collaborative live streaming of a free viewpoint video, where a group of users may interactively pull and cooperatively share streams of different anchor views. There is a cost to access the anchor views from the live source, a cost to "reconfigure" the peer network due to a change in selected anchors during view switching, and a distortion cost due to the distance of the virtual views to the received anchor views at users. We optimize the anchor views allocated to users so as to minimize the overall streaming cost given by the access cost, reconfiguration cost, and view distortion cost. We first show that, if the reconfiguration cost due to view switching is negligible, the view allocation problem can be optimally and efficiently solved in polynomial time using dynamic programming. For the case of non-negligible reconfiguration cost, the problem becomes NP-hard. We thus present a locally optimal and centralized algorithm inspired by Lloyd's algorithm used in non-uniform scalar quantization. We further propose a distributed algorithm with convergence guarantee, where each peer group independently makes merge-and-split decisions with a well-defined fairness criteria. Simulation results show that our algorithms achieve low streaming cost due to its excellent anchor view allocation.

show abstract

Section: A Multiview Video Streaming and High-dimensional Media Navimentioning

confidence: 99%

Section: B Experiments and Simulationmentioning

confidence: 99%

Section: B Experiments and Simulationmentioning

confidence: 99%

Section: B Experiments and Simulationmentioning

confidence: 99%

See 2 more Smart Citations

Anchor View Allocation for Collaborative Free Viewpoint Video Streaming

Ren

Chan

Cheung

et al. 2015

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…It has been shown [7] experimentally that WPB performs well when packet losses across views are independent.…”

Section: Introductionmentioning

confidence: 99%

Saliency-cognizant robust view synthesis in free viewpoint video streaming

Macchiavello

Dorea

Hung

et al. 2013

2013 IEEE International Conference on Image Processing

Self Cite

View full text Add to dashboard Cite

In free viewpoint video, texture and depth maps from two cameracaptured viewpoints are transmitted, so that at receiver, a novel virtual view chosen by the client can be synthesized via depthimage-based rendering (DIBR). When irrecoverable packet losses occur during transmission-typically affecting less important spatial regions in the video given unequal error protection (UEP) is deployed-appropriate error concealment strategies must be used at decoder to minimize resulting visual degradation in the synthesized view. Towards this goal, we propose a new optimization framework based on visual saliency to combine two different concealment techniques. First, given a pixel in the virtual view is typically constructed as a convex combination of corresponding pixels in the left and right captured views, weighted pixel blending (WPB) readjusts the weights in the linear sum to reflect the expected error in code blocks that contain the corresponding pixels. Second, exemplarbased patch matching (EPM) finds the most similar patches in the known spatial region to complete missing pixels in the unknown region. To choose between candidates constructed using the two techniques when filling a given pixel patch in the synthesized view, we first compute a weighted sum of expected error and visual saliency for each candidate patch. The candidate with the smaller sum (one with small expected error and visual saliency, so that even if errors do occur, they do not stand out visually) is selected for pixel completion. Experimental results show that our scheme can outperform the use of co-located blocks from a previous frame by up to 0.7dB in PSNR and improve subjective visual quality.Index Terms-free viewpoint video, visual saliency, error concealment

show abstract

Optimizing peer grouping for live free viewpoint video streaming

Yuan

Cheung

et al. 2013

2013 IEEE International Conference on Image Processing

Self Cite

View full text Add to dashboard Cite

In free viewpoint video, a user can pull texture and depth videos captured from two nearby reference viewpoints to synthesize his chosen intermediate virtual view for observation via depth-image-based rendering (DIBR). For users who are observing the same video at the same time but not necessarily from the same virtual viewpoint, they have incentive to pull the same reference views so that the streaming cost can be shared. On the other hand, in general distortion of a synthesized virtual view increases with its distance to the reference views, and so a user also has incentive to select reference views that tightly "sandwich" his chosen virtual view, minimizing distortion. In a previous work, reference view sharing strategiesones that optimally trade off shared streaming costs with synthesized view distortions-were investigated for the case when users are first divided into groups, and each user group independently pulls two reference views and shares the resulting streaming cost. In this paper, we generalize the previous notion of user group, so that a user can simultaneously belong to two groups, and each group shares the streaming cost of a single view. We also aim to find a Nash Equilibrium (NE) solution of reference view selection, which is stable and from which no one has incentive to unilaterally deviate. Specifically, we first derive a lemma based on known properties of synthesized view distortion functions. We then design a search algorithm to find a NE solution, leveraging on the derived lemma to reduce search complexity. Experimental results show that the stable NE solution increases the overall cost only slightly when compared to the unstable optimal reference selection that gives the lowest overall cost. Further, a larger network will give a lower average cost for each user, and thus, users tend to join large networks for cooperation.

show abstract

Reference frame selection for loss-resilient texture & depth map coding in multiview video conferencing

Cited by 12 publications

References 13 publications

Anchor View Allocation for Collaborative Free Viewpoint Video Streaming

Anchor View Allocation for Collaborative Free Viewpoint Video Streaming

Saliency-cognizant robust view synthesis in free viewpoint video streaming

Optimizing peer grouping for live free viewpoint video streaming

Contact Info

Product

Resources

About