Exploiting color-depth image correlation to improve depth map compression

Abstract

Abstract-The multimedia signal processing community has recently identified the need to design depth map compression algorithms which preserve depth discontinuities in order to improve the rendering quality of virtual views for Free Viewpoint Video (FVV) services. This paper adopts contour detection with surround suppression on the color video to approximate the foreground edges present in the depth image. Displacement estimation and compensation is then used to improve this prediction and reduce the amount of side information required by the decoder. Simulation results indicate that the proposed method manages to accurately predict around 64% of the blocks. Moreover, the proposed scheme achieves a Peak Signal-to-Noise Ratio (PSNR) gain of around 4.9 -6.6 dB relative to the JPEG standard and manages to outperform other state of the art depth map compression algorithms found in literature.Index Terms-3D television, depth map compression, displacement estimation and compensation, edge detection, surround suppression I. INTRODUCTION Advances in technology have contributed to the development of new applications such as 3D Television (3DTV) and Free Viewpoint Video (FVV), which have recently started to attract interest within the marketplace. The multi-view video plus depth (MVD) [1] format enables the generation of an infinite set of videos using a finite set of color and corresponding depth videos [2]. The MVD format significantly reduces the amount of information needed to deploy these services. Moreover, in order to make these applications more viable, both color and depth videos need to be compressed. However, the quality of the rendered views is significantly dependent on the quality of the depth video.Traditional image and video compression standards produce blurring artifacts along depth discontinuities, which negatively affect the Quality of Experience (QoE) of the rendered virtual views [3], [4]. The authors in [5] have demonstrated that higher rendering quality can be perceived when employing depth map compression strategies which preserve edge boundaries. Following this observation the same authors have proposed to reshape the dynamic range and use Region of Interest (RoI) coding to improve the performance of traditional image compression standards. Directional transforms were considered in [3], [6] which are highly computational intensive.

Compression of Noisy Depth Image using Planes

“…Farrugia and Gambin [3] extends the method presented in [2] to exploit the existed inherent correlation between the color image and the corresponding depth image. They adopt contour detection with surround suppression to approximate the edges in the depth map using the color video.…”

“…Many of them rely on piece-wise linear regions and start with image segmentation. Sometimes the associated color image can aid in this segmentation [12] [3].…”