Fast disparity estimation for Multi-view plus depth video coding

Abstract

Disparity estimation is used in Multi-view Video Coding (MVC) to remove the inter-view redundancies present in both color and depth multi-view video sequences. The standard H.264/MVC achieves high compression efficiency by deriving the optimal disparity vector through the exhaustive calculation of the Rate-Distortion cost function for all the possible search points. This makes disparity estimation highly computational expensive. This paper proposes an efficient technique that exploits both the multi-view and the epipolar geometries to determine the optimal search area, resulting in a reduction of search points and thus computations. Simulation results show that this technique can save up to 95% of the computational cost for disparity estimation, with negligible loss in coding efficiency for both the color and the depth multi-view video coding.

Low complexity disparity estimation for immersive 3D video transmission

“…This is based on the fact that depth and disparity are directly proportional, thus, the higher the depth change, the higher the expected disparity and the larger the required DE's search area. This adaptive technique outperforms the speedup gain of our previous work [9], by about 1.5 times, where a fixed search area along the epipolar lines was considered. Moreover, combining these techniques we achieve a speedup gain of up-to about 32 times over the disparity estimation technique adopted by the standard H.264-MVC.…”

“…The smallest distortions are normally obtained from the corresponding multi-view areas, as they represent the same object in the viewpoint reference frames [17]. Hence, usually searching only around these similar areas can significantly reduce the exhaustive search and its computational time [9,[18][19][20][21]. These areas can be identified through the multi-view geometry [17] and the average depth value of the current sub-MB to estimate, with the depth values obtained from the depth map MVV.…”

“…Since the majority of the depth pixel elements (pels) bias the average depth value, this can provide an accurate compensation area to estimate the sub-MB pels. Nevertheless, a significantly smaller search area of ±10 pels [9,18] is still required to identify an optimal estimate for the whole block, as the average depth can only estimate a large part of its pels, but not necessarily the entire sub-MB. Thus, it only provides a good position where to search.…”

“…Furthermore, the DE will normally identify as optimal replacements a block along the epipolar lines [14], as these lines represent the relationship between the multi-view images [17]. Thus, our work in [9] proposed the use of the multi-view geometry to obtain the corresponding multi-view areas, but then search only along the epipolar lines, to reduce further the computational cost. Even so, a search area of ±10 pels, along these lines, was still maintained as the average depth value remained imprecise to obtain an accurate compensation match for the entire block, but the search area above and below these geometric lines was reduced to 3 pixel elements.…”

“…To reduce these computations, in our previous work, we proposed a solution that exploited the rich geometrical depth data provided through the MVD representation, together with the multi-view and the epipolar geometries, to accurately estimate an adequate fixed search area along the epipolar lines, for disparity estimation [9]. In this paper, we extend this idea and propose an adaptive search area along these geometric lines, which is proportional to the largest depth variation within the sub-Macro-block (MB) to encode.…”