Dense motion estimation between distant frames: Combinatorial multi-step integration and statistical selection

Abstract

Accurate estimation of dense point correspondences between two distant frames of a video sequence is a challenging task. To address this problem, we present a combinatorial multistep integration procedure which allows one to obtain a large set of candidate motion fields between the two distant frames by considering multiple motion paths across the video sequence. Given this large candidate set, we propose to perform the optimal motion vector selection by combining a global optimization stage with a new statistical processing. Instead of considering a selection only based on intrinsic motion field quality and spatial regularization, the statistical processing exploits the spatial distribution of candidates and introduces an intra-candidate quality based on forward-backward consistency. Experiments evaluate the effectiveness of our method for distant motion estimation in the context of video editing.

Dense Long-term Motion Estimation via Statistical Multi-step Flow

“…Compared to (Conze et al, 2013), our displacement fields selection procedure combines differently statistical selection and global optimization. For each x re f ∈ I re f , we select among T re f ,n (x re f ) K sp = 2 × K candidates through statistical selection, with K sp < K x re f .…”

“…Our explanations focus on the estimation of from-thereference displacement fields. In the following, we describe the input data and recall the baseline method (Conze et al, 2013) before focusing on how it has been improved and extended to the whole sequence.…”

“…These optical flows are previously estimated between consecutive frames or with larger This combinatorial integration (Conze et al, 2013) is done independently for each pair {I re f , I n } which limits the correlation between candidates selected for neighbouring frames.…”

“…These limitations can be resolved by extending to the whole sequence the combinatorial multi-step integration and the statistical selection described in (Conze et al, 2013) for dense motion estimation between a pair of distant frames. The underlying idea is to first consider a large set composed of combinations of multi-step optical flows and then to study the spatial redundancy of the resulting candidates through a statistical selection to finally select the best matches.…”

“…First, it extends (Conze et al, 2013) to generate several initial dense correspondences between the reference frame and each of the subsequent images independently. Second, we propose to provide an accurate final dense matching by applying a new iterative motion refinement which involves strong temporal smoothness constraints.…”