Performance of scalable coding in depth domain

Abstract

Common autostereoscopic 3D displays are based on multi-view projection. The diversity of resolutions and number of views of such displays implies a necessary flexibility of 3D content formats in order to make broadcasting efficient. Furthermore, distribution of content over a heterogeneous network should adapt to an available network capacity. Present scalable video coding provides the ability to adapt to network conditions; it allows for quality, temporal and spatial scaling of 2D video. Scalability for 3D data extends this list to the depth and the view domains. We have introduced scalability with respect to depth information. Our proposed scheme is based on the multi-view-plus-depth format where the center view data are preserved, and side views are extracted in enhancement layers depending on depth values. We investigate the performance of various layer assignment strategies: number of layers, and distribution of layers in depth, either based on equal number of pixels or histogram characteristics. We further consider the consequences to variable distortion due to encoder parameters. The results are evaluated considering their overall distortion verses bit rate, distortion per enhancement layer, as well as visual quality appearance. Scalability with respect to depth (and views) allows for an increased number of quality steps; the cost is a slight increase of required capacity for the whole sequence. The main advantage is, however, an improved quality for objects close to the viewer, even if overall quality is worse.

Layer Assignment Based on Depth Data Distribution for Multiview-Plus-Depth Scalable Video Coding

“…The present work is an extension to authors' previous works [20] and [21], where an approach depth and view scalability of a MVD sequence of three views was proposed. The depth scalability concerns scalability of the color data in relation to the distance to the camera.…”

“…However, the segmentation of objects increases the complexity of the algorithm substantially and may introduce errors. The layer assignment is therefore performed using the depth distribution layer assignment scheme (DLA) that was first proposed by the authors in [21] using the name Scheme C. DLA utilizes the characteristics of the depth data distribution both to determine the position in depth and the number of enhancement layers. DLA was shown in [21] to provide a improved performance compared to using layer assignment schemes that are based on a uniform distribution of the pixels over the enhancement layers.…”