Occlusion Aware Unsupervised Learning of Optical Flow

Wang, Yan; Yang, Yi; Yang, Zhenheng; Zhao, Liang; Wang, Peng; Xu, Wei

doi:10.1109/cvpr.2018.00513

Cited by 322 publications

(301 citation statements)

References 55 publications

(78 reference statements)

Supporting

Mentioning

292

Contrasting

Order By: Relevance

“…Smoothness Loss. For the smoothness loss L sm , we adopt the formulation introduced in [25] which encourages the correspondence maps to be locally smooth but also maintains edges that should be aligned with the structure of images:…”

Section: Unsupervised Loss Functionsmentioning

confidence: 99%

Bridging Stereo Matching and Optical Flow via Spatiotemporal Correspondence

Lai

Tsai

Chiu

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Stereo matching and flow estimation are two essential tasks for scene understanding, spatially in 3D and temporally in motion. Existing approaches have been focused on the unsupervised setting due to the limited resource to obtain the large-scale ground truth data. To construct a self-learnable objective, co-related tasks are often linked together to form a joint framework. However, the prior work usually utilizes independent networks for each task, thus not allowing to learn shared feature representations across models. In this paper, we propose a single and principled network to jointly learn spatiotemporal correspondence for stereo matching and flow estimation, with a newly designed geometric connection as the unsupervised signal for temporally adjacent stereo pairs. We show that our method performs favorably against several state-of-the-art baselines for both unsupervised depth and flow estimation on the KITTI benchmark dataset. arXiv:1905.09265v1 [cs.CV] 22 May 2019 r,t = W (I l,t , D r,t→l,t ),

show abstract

Section: Unsupervised Loss Functionsmentioning

confidence: 99%

Bridging Stereo Matching and Optical Flow via Spatiotemporal Correspondence

Lai

Tsai

Chiu

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…We use a grid search to set the balance parameters, λ D , λ O and λ, to 0.1, 0.1 and 0.05, respectively. We follow the experimental setting in [12], [16], [49], [60] to set other parameters, and fix them in all experiment: α = 0.5, β = 0.85, γ = 10, and = 1. We compute optical flow using the DIS-Flow method [61] that offers a good compromise in terms of runtime and accuracy.…”

Section: A Trainingmentioning

confidence: 99%

Temporally Consistent Depth Prediction With Flow-Guided Memory Units

Eom

Park

Ham

2020

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Predicting depth from a monocular video sequence is an important task for autonomous driving. Although it has advanced considerably in the past few years, recent methods based on convolutional neural networks (CNNs) discard temporal coherence in the video sequence and estimate depth independently for each frame, which often leads to undesired inconsistent results over time. To address this problem, we propose to memorize temporal consistency in the video sequence, and leverage it for the task of depth prediction. To this end, we introduce a two-stream CNN with a flow-guided memory module, where each stream encodes visual and temporal features, respectively. The memory module, implemented using convolutional gated recurrent units (ConvGRUs), inputs visual and temporal features sequentially together with optical flow tailored to our task. It memorizes trajectories of individual features selectively and propagates spatial information over time, enforcing a long-term temporal consistency to prediction results. We evaluate our method on the KITTI benchmark dataset in terms of depth prediction accuracy, temporal consistency and runtime, and achieve a new state of the art. We also provide an extensive experimental analysis, clearly demonstrating the effectiveness of our approach to memorizing temporal consistency for depth prediction.

show abstract

“…We found that a photoconsistency loss on the reblurred images alone is insufficient to constrain the optical flow. We thus add an additional self-supervised photometric loss on the optical flow as proposed in prior work [12], [20], [34] and detailed in Section III-D. The input to this loss function is a Note that bilinear interpolation cannot be applied since the warped grid might not be rectangular due to the non-uniform optical flow, as shown in Fig.…”

Section: Image Warpingmentioning

confidence: 99%

Self-Supervised Linear Motion Deblurring

Liu

Janai

Pollefeys

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Motion blurry images challenge many computer vision algorithms, e.g., feature detection, motion estimation, or object recognition. Deep convolutional neural networks are stateof-the-art for image deblurring. However, obtaining training data with corresponding sharp and blurry image pairs can be difficult. In this paper, we present a differentiable reblur model for selfsupervised motion deblurring, which enables the network to learn from real-world blurry image sequences without relying on sharp images for supervision. Our key insight is that motion cues obtained from consecutive images yield sufficient information to inform the deblurring task. We therefore formulate deblurring as an inverse rendering problem, taking into account the physical image formation process: we first predict two deblurred images from which we estimate the corresponding optical flow. Using these predictions, we re-render the blurred images and minimize the difference with respect to the original blurry inputs. We use both synthetic and real dataset for experimental evaluations. Our experiments demonstrate that self-supervised single image deblurring is really feasible and leads to visually compelling results. Both the code and datasets are available at https://github.com/ethliup/SelfDeblur.

show abstract

Occlusion Aware Unsupervised Learning of Optical Flow

Cited by 322 publications

References 55 publications

Bridging Stereo Matching and Optical Flow via Spatiotemporal Correspondence

Bridging Stereo Matching and Optical Flow via Spatiotemporal Correspondence

Temporally Consistent Depth Prediction With Flow-Guided Memory Units

Self-Supervised Linear Motion Deblurring

Contact Info

Product

Resources

About