Depth map compression using multi-resolution graph-based transform for depth-image-based rendering

Abstract

Depth map compression is important for efficient network transmission of 3D visual data in texture-plus-depth format, where the observer can synthesize an image of a freely chosen viewpoint via depth-image-based rendering (DIBR) using received neighboring texture and depth maps as anchors. Unlike texture maps, depth maps exhibit unique characteristics like smooth interior surfaces and sharp edges that can be exploited for coding gain. In this paper, we propose a multi-resolution approach to depth map compression using previously proposed graph-based transform (GBT). The key idea is to treat smooth surfaces and sharp edges of large code blocks separately and encode them in different resolutions: encode edges in original high resolution (HR) to preserve sharpness, and encode smooth surfaces in low-pass-filtered and down-sampled low resolution (LR) to save coding bits. Because GBT does not filter across edges, it produces small or zero high-frequency components when coding smooth-surface depth maps and leads to a compact representation in the transform domain. By encoding down-sampled surface regions in LR GBT, we achieve representation compactness for a large block without the high computation complexity associated with an adaptive large-block GBT. At the decoder, encoded LR surfaces are up-sampled and interpolated while preserving encoded HR edges. Experimental results show that our proposed multi-resolution approach using GBT reduced bitrate by 68% compared to native H.264 intra with DCT encoding original HR depth maps, and by 55% compared to single-resolution GBT encoding small blocks.

Image interpolation for DIBR viewsynthesis using graph fourier transform

“…In particular, GFT has been successfully used for depth map compression [9], denoising [10], etc. In this paper, we propose to use GFT for expansion hole filling, or more generally, image interpolation.…”

Expansion hole filling in depth-image-based rendering using graph-based interpolation

“…Compared to non-local image interpolation methods [15], the complexity of our GBT interpolation is bounded by the few number of pixels within the neighborhood of an expansion hole used to construct the graph. While GBT has been used for compression of depth maps [16,17], this is the first work in the literature of using GBT for image patch interpolation.…”

A graph-based signal processing approach for low-rate energy disaggregation

“…Indeed, in [16] potentials of GSP to linear predication, customer behavior prediction, and image compression are demonstrated. GSP, or more specifically, graph Fourier transform, has been used for image compression (depth map coding) and image denoising in [19] and [20], respectively. In [17], the GSP tools are used for dataset classification, where it is shown that the GSP-based classification, as applied to image classification, provides more accurate and more robust results compared to standard support vector machine (SVM) and neural network-based approaches.…”