Accurate Optical Flow via Direct Cost Volume Processing

Abstract

We present an optical flow estimation approach that operates on the full four-dimensional cost volume. This direct approach shares the structural benefits of leading stereo matching pipelines, which are known to yield high accuracy. To this day, such approaches have been considered impractical due to the size of the cost volume. We show that the full four-dimensional cost volume can be constructed in a fraction of a second due to its regularity. We then exploit this regularity further by adapting semi-global matching to the four-dimensional setting. This yields a pipeline that achieves significantly higher accuracy than state-of-the-art optical flow methods while being faster than most. Our approach outperforms all published general-purpose optical flow methods on both Sintel and KITTI 2015 benchmarks.

Sparse Data Interpolation Using the Geodesic Distance Affinity Space

“…Second, we analyze the advantage of our approach over the EFI method [19] for the optical flow problem. In our experiments, we used the MPI Sintel data set [6] and the fast version of the DCflow [26] method pipeline.…”

“…Finally, the proposed approach faster, more general and with clearer theoretical motivation than the baseline algorithm [19]. We applied our interpolation method to the sparse optical flow data obtained by the DCflow [26] method and compared with the interpolation result of the interpolation in [19] on the same sparse data set. Formally we included our interpolation method in the pipeline of the DCflow [26] method and compared it with the result of the same DCflow pipeline that included the EpicFlow interpolation (EFI) instead ours.…”

“…We applied our interpolation method to the sparse optical flow data obtained by the DCflow [26] method and compared with the interpolation result of the interpolation in [19] on the same sparse data set. Formally we included our interpolation method in the pipeline of the DCflow [26] method and compared it with the result of the same DCflow pipeline that included the EpicFlow interpolation (EFI) instead ours. The comparison shows that our algorithm makes the fast version of the DCflow [26] pipeline more accurate than the EFI technique while being considerably faster.…”

“…Formally we included our interpolation method in the pipeline of the DCflow [26] method and compared it with the result of the same DCflow pipeline that included the EpicFlow interpolation (EFI) instead ours. The comparison shows that our algorithm makes the fast version of the DCflow [26] pipeline more accurate than the EFI technique while being considerably faster.…”

U4D: Unsupervised 4D Dynamic Scene Understanding

“…Flow from the joint estimation is evaluated against stateof-the-art methods: (a) Dense flow algorithms DCflow [57] and Deepflow [54]; (b) Scene flow methods PRSM [52]; and (c) Non-sequential alignment of partial surfaces 4DMatch [38] (requires a prior 3D mesh of the object as input for 4D reconstruction). The key-frames of sequence are coloured and the colour is propagated using dense flow from the joint optimisation throughout the sequence.…”