Fast Guided Global Interpolation for Depth and Motion

Liu, Yu; Min, Dongbo; Minh, N.; Lu, Jiangbo

doi:10.1007/978-3-319-46487-9_44

Cited by 68 publications

(67 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Occlusion has been regarded as an important cue for estimating more accurate optical flow. Because occluded pixels do not have correspondences in the other frame, several approaches [5,12,16,23,32] aim to filter out these outliers to minimize their ill effects and apply post-processing to refine the estimates [22,33,48,67].…”

Section: Related Workmentioning

confidence: 99%

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation

Hur

Roth

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

280

239

View full text Add to dashboard Cite

Deep learning approaches to optical flow estimation have seen rapid progress over the recent years. One common trait of many networks is that they refine an initial flow estimate either through multiple stages or across the levels of a coarse-to-fine representation. While leading to more accurate results, the downside of this is an increased number of parameters. Taking inspiration from both classical energy minimization approaches as well as residual networks, we propose an iterative residual refinement (IRR) scheme based on weight sharing that can be combined with several backbone networks. It reduces the number of parameters, improves the accuracy, or even achieves both. Moreover, we show that integrating occlusion prediction and bi-directional flow estimation into our IRR scheme can further boost the accuracy. Our full network achieves stateof-the-art results for both optical flow and occlusion estimation across several standard datasets. Number of parameters (million) AEPE on Sintel Train Clean FlowNetC FlowNetS SpyNetOurs (PWC-Net + Occ) PWC-NetOurs (PWC-Net + Bi) Ours (IRR-PWC) LiteFlowNetOurs (PWC-Net + IRR)

show abstract

Section: Related Workmentioning

confidence: 99%

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation

Hur

Roth

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

280

239

View full text Add to dashboard Cite

show abstract

“…2a. With the aid of highly capable interpolation methods [26,31], this pipeline has been regarded as a well-justified practice. However, occasional failures during post-processing are inevitable and irreversible, and thus constitute a fundamental limitation.…”

Section: Related Workmentioning

confidence: 99%

MirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation

Hur

Roth

2017

2017 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Figure 1. Results of our symmetric optical flow approach given two consecutive images from the KITTI benchmark [12]. Our method jointly predicts forward and backward optical flow (overlaid on top row, from left to right, color coding of the benchmark) as well as corresponding occlusion maps for each view (overlaid on bottom row). AbstractOptical flow estimation is one of the most studied problems in computer vision, yet recent benchmark datasets continue to reveal problem areas of today's approaches. Occlusions have remained one of the key challenges. In this paper, we propose a symmetric optical flow method to address the well-known chicken-and-egg relation between optical flow and occlusions. In contrast to many state-ofthe-art methods that consider occlusions as outliers, possibly filtered out during post-processing, we highlight the importance of joint occlusion reasoning in the optimization and show how to utilize occlusion as an important cue for estimating optical flow. The key feature of our model is to fully exploit the symmetry properties that characterize optical flow and occlusions in the two consecutive images. Specifically through utilizing forward-backward consistency and occlusion-disocclusion symmetry in the energy, our model jointly estimates optical flow in both forward and backward direction, as well as consistent occlusion maps in both views. We demonstrate significant performance benefits on standard benchmarks, especially from the occlusiondisocclusion symmetry. On the challenging KITTI dataset we report the most accurate two-frame results to date.Given the two consecutive images I t and I t+1 with their superpixel representations, our model jointly estimates (i)

show abstract

“…(II) The FGS only adopts the 4-connected/8connected neighborhood system while our method can handle a more general (2r + 1) × (2r + 1) neighborhood system thanks to the proposed fast solution. We show that such a generalization makes our method achieve better performance than the 4-connected/8-connected one in some applications such as guided depth upsampling [7,16].…”

Section: Background and Related Workmentioning

confidence: 93%

Semi-Global Weighted Least Squares in Image Filtering

Liu

Chen

Shen

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Solving the global method of Weighted Least Squares (WLS) model in image filtering is both time-and memory-consuming. In this paper, we present an alternative approximation in a time-and memory-efficient manner which is denoted as Semi-Global Weighed Least Squares (SG-WLS). Instead of solving a large linear system, we propose to iteratively solve a sequence of subsystems which are one-dimensional WLS models. Although each subsystem is one-dimensional, it can take two-dimensional neighborhood information into account due to the proposed special neighborhood construction. We show such a desirable property makes our SG-WLS achieve close performance to the original two-dimensional WLS model but with much less time and memory cost. While previous related methods mainly focus on the 4-connected/8-connected neighborhood system, our SG-WLS can handle a more general and larger neighborhood system thanks to the proposed fast solution. We show such a generalization can achieve better performance than the 4-connected/8connected neighborhood system in some applications. Our SG-WLS is ∼ 20 times faster than the WLS model. For an image of M ×N , the memory cost of SG-WLS is at most at the magnitude of max{ 1 M , 1 N } of that of the WLS model. We show the effectiveness and efficiency of our SG-WLS in a range of applications.

show abstract

Fast Guided Global Interpolation for Depth and Motion

Cited by 68 publications

References 33 publications

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation

MirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation

Semi-Global Weighted Least Squares in Image Filtering

Contact Info

Product

Resources

About