Evaluation of depth compression and view synthesis distortions in multiview-video-plus-depth coding systems

Abstract

Several quality evaluation studies have been performed for videoplus-depth coding systems. In these studies, however, the distortions in the synthesized views have been quantified in experimental setups where both the texture and depth videos are compressed. Nevertheless, there are several factors that affect the quality of the synthesized view. Incorporating more than one source of distortion in the study could be misleading; one source of distortion could mask (or be masked by) the effect of other sources of distortion. In this paper, we conduct a quality evaluation study that aims to assess the distortions introduced by the view synthesis procedure and depth map compression in multiview-video-plus-depth coding systems. We report important findings that many of the existing studies have overlooked, yet are essential to the reliability of quality evaluation. In particular, we show that the view synthesis reference software yields high distortions that mask those due to depth map compression, when the distortion is measured by average luma peak signalto-noise ratio. In addition, we show what quality metric to use in order to reliably quantify the effect of depth map compression on view synthesis quality. Experimental results that support these findings are provided for both synthetic and real multiview-video-plusdepth sequences.Index Terms -Depth map compression, multi-view video coding, video plus depth, view synthesis

Can 3D synthesized views be reliably assessed through usual subjective and objective evaluation protocols?

“…Though, the subjective scores correlated the VSSIM scores. Recently, [6] reconsidered the synthesis quality evaluation framework. The authors showed the importance of the chosen reference for synthesis quality evaluation: they pointed out the fact that depending on the chosen reference (original view or control synthesis, i.e.…”

Efficient Techniques for Depth Video Compression Using Weighted Mode Filtering

“…For instance, there are a lot of distortion sources such as slightly different camera response function, exposure time, and lighting condition among multiview images. It has also been shown in [34] that the distortion introduced by VSRS widely masks those due to depth map compression, which can result in a misleading study in order to justify the effectiveness of depth compression.…”

Evaluating multi-view plus depth coding solutions for 3D video scenarios

“…In the past, a detailed evaluation of multi-view video coding schemes has been made, mainly focusing on the extension of the popular H.264/AVC coding standard [4]. More recently, some preliminary evaluation of depth map compression performance was made in [5,6]. While [5] evaluates the coding distortion of two depth map coding solutions, notably H.264/AVC and a depthimage modeling approach, [6] assesses the performance of the H.264/AVC video coding standard using three PSNR-based quality metrics.…”

“…More recently, some preliminary evaluation of depth map compression performance was made in [5,6]. While [5] evaluates the coding distortion of two depth map coding solutions, notably H.264/AVC and a depthimage modeling approach, [6] assesses the performance of the H.264/AVC video coding standard using three PSNR-based quality metrics. However, in both studies, only the H.264/AVC coding standard is considered and uncoded original views (texture) are used at the decoder side which is not a very realistic situation.…”

“…Then, the bit streams are transmitted to the decoder assuming a noiseless transmission channel.  Decoding step: At the decoder, the reconstructed texture and depth sequences (for anchor views) are obtained and used as input to a DIBR module [9] able to synthesize the intermediate view C between the two input decoded views L and R. Notice that, as opposed to past studies where original views were used [5,6], here the texture has quantization errors (as also the depth maps) that may affect the quality of the synthesized view C.  Quality evaluation: To evaluate the coding performance, a reference synthesized view is obtained by applying the same DIBR technique while using original texture and depth maps. In this context, it is possible to assess the quality of the synthesized intermediate view using the synthesized intermediate view as reference and a PSNR quality metric as done by the MPEG FTV Ad-Hoc group.…”