Disparity-compensated view synthesis for S3D content correction

Abstract

The production of stereoscopic 3D HD content is considerably increasing and experience in 2-view acquisition is in progress. High quality material to the audience is required but not always ensured, and correction of the stereo views may be required. This is done via disparity-compensated view synthesis. A robust method has been developed dealing with these acquisition problems that introduce discomfort (e.g hyperdivergence and hyperconvergence…) as well as those ones that may disrupt the correction itself (vertical disparity, color difference between views…). The method has three phases: a preprocessing in order to correct the stereo images and estimate features (e.g. disparity range…) over the sequence. The second (main) phase proceeds then to disparity estimation and view synthesis. Dual disparity estimation based on robust block-matching, discontinuity-preserving filtering, consistency and occlusion handling has been developed. Accurate view synthesis is carried out through disparity compensation. Disparity assessment has been introduced in order to detect and quantify errors. A post-processing deals with these errors as a fallback mode. The paper focuses on disparity estimation and view synthesis of HD images. Quality assessment of synthesized views on a large set of HD video data has proved the effectiveness of our method.

Dense motion estimation between distant frames: Combinatorial multi-step integration and statistical selection

“…For the selected pairs, the combinatorial multi-step integration has been performed taking input elementary flow fields estimated with a 2D version of the disparity estimator of [17]. For all the experiments, the parameters are : N c = 7, N s = 100, Q max = 2, N opt = 3.…”

Dense Long-term Motion Estimation via Statistical Multi-step Flow

“…, N − n}, the occlusion mask attached to the optical flow field v n,n+s i indicates the visibility of each pixel of I n in I n+s i . The inconsistency mask attached to v n,n+s i distinguishes consistent and inconsistent optical flow vectors among the ones starting from pixels marked as visible (Robert et al, 2012). This feature follows the idea that the backward flow should be the exact opposite of the forward flow.…”

“…This proves that StatFlow is competitive compared to challenging state-of-the-art methods. (Brox and Malik, 2011) optical flow fields computed between consecutive frames (LFOF acc); g-i) multi-step flow fusion (MSF) (Crivelli et al, 2012a) using multi-step optical flow fields from (Robert et al, 2012) (2D-DE); j-l) the proposed statistical multi-step flow (StatFlow) using 2D-DE multi-step optical flow fields. Figure 8: Source frames of the Flag sequence (Garg et al, 2013).…”

“…Hope and Newspaper) sequence up to I 137 (resp. I 5063 and I 170 ) using: 1) multi-step flow fusion (MSF) (Crivelli et al, 2012a) with multi-step optical flow fields from (Robert et al, 2012) …”

“…The sets of steps are 1 − 5, 10 (NP), 15 (MPI S1), 20 (NP) and 30 (NP). The algorithms are performed taking input multi-step optical flows computed with a 2D version of the disparity estimator described in (Robert et al, 2012), referred to as 2D-DE.…”