What Matters in Unsupervised Optical Flow

Jonschkowski, Rico; Stone, Austin; Barron, Jonathan T.; Gordon, Aaron; Konolige, Kurt; Angelova, Anelia

doi:10.1007/978-3-030-58536-5_33

Cited by 128 publications

(122 citation statements)

References 31 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…We conduct this experiment on top of our multiframe architecture. Our occlusion-aware census further improves the scene flow accuracy by 6.0% (relative improvement) over the basic photometric loss and by 4.5% over the standard census transform [28].…”

Section: Multi-frame Extensionmentioning

confidence: 95%

“…Carefully designing the proxy loss function matters for the accuracy of selfsupervised learning [28]. For penalizing the photometric difference for the view-synthesis proxy task, the census transform [60,76] has demonstrated its robustness to illumination changes, e.g., in outdoor scenes [28,33,41,66]. The conventional (ternary) census transform computes the local census patch (Fig.…”

Section: Self-supervised Lossmentioning

confidence: 99%

“…Occ-aware census loss D1-all D2-all Fl-all SF-all [28] and use one less pyramid level and feature normalization, which further improves the accuracy, even slightly beyond that of the original baseline trained on the split of [23] (covering more diverse scenes).…”

Section: Multi-frame Extensionmentioning

confidence: 99%

“…Jonschkowski et al [28] provide a systematical analysis of the key design factors for highly accurate self-supervised optical flow. We conduct an empirical study on which of their key findings are beneficial in the context of monocular 3D scene flow.…”

Section: B Refined Backbone Architecturementioning

confidence: 99%

See 3 more Smart Citations

Self-Supervised Monocular Scene Flow Estimation

Hur

Roth

2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

105

View full text Add to dashboard Cite

Estimating 3D scene flow from a sequence of monocular images has been gaining increased attention due to the simple, economical capture setup. Owing to the severe illposedness of the problem, the accuracy of current methods has been limited, especially that of efficient, real-time approaches. In this paper, we introduce a multi-frame monocular scene flow network based on self-supervised learning, improving the accuracy over previous networks while retaining real-time efficiency. Based on an advanced twoframe baseline with a split-decoder design, we propose (i) a multi-frame model using a triple frame input and convolutional LSTM connections, (ii) an occlusion-aware census loss for better accuracy, and (iii) a gradient detaching strategy to improve training stability. On the KITTI dataset, we observe state-of-the-art accuracy among monocular scene flow methods based on self-supervised learning.

show abstract

Section: Multi-frame Extensionmentioning

confidence: 95%

Section: Self-supervised Lossmentioning

confidence: 99%

Section: Multi-frame Extensionmentioning

confidence: 99%

Section: B Refined Backbone Architecturementioning

confidence: 99%

See 2 more Smart Citations

Self-Supervised Monocular Scene Flow Estimation

Hur

Roth

2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

105

View full text Add to dashboard Cite

show abstract

“…Moreover, we use an edge-aware smoothness loss [61], [62] to regularize the estimated disparity map, i.e.,…”

Section: Warp-netmentioning

confidence: 99%

Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses

Jin

Hou

Chen

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

This paper explores the problem of reconstructing high-resolution light field (LF) images from hybrid lenses, including a high-resolution camera surrounded by multiple lowresolution cameras. The performance of existing methods is still limited, as they produce either blurry results on plain textured areas or distortions around depth discontinuous boundaries. To tackle this challenge, we propose a novel end-to-end learningbased approach, which can comprehensively utilize the specific characteristics of the input from two complementary and parallel perspectives. Specifically, one module regresses a spatially consistent intermediate estimation by learning a deep multidimensional and cross-domain feature representation, while the other module warps another intermediate estimation, which maintains the high-frequency textures, by propagating the information of the high-resolution view. We finally leverage the advantages of the two intermediate estimations adaptively via the learned attention maps, leading to the final high-resolution LF image with satisfactory results on both plain textured areas and depth discontinuous boundaries. Besides, to promote the effectiveness of our method trained with simulated hybrid data on real hybrid data captured by a hybrid LF imaging system, we carefully design the network architecture and the training strategy. Extensive experiments on both real and simulated hybrid data demonstrate the significant superiority of our approach over state-of-the-art ones. To the best of our knowledge, this is the first end-to-end deep learning method for LF reconstruction from a real hybrid input. We believe our framework could potentially decrease the cost of high-resolution LF data acquisition and benefit LF data storage and transmission. The code will be publicly available at https://github.com/jingjin25/LFhybridSR-Fusion.

show abstract

BMBC: Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation

Park¹,

Ko²,

Lee

et al. 2020

Computer Vision – ECCV 2020

131

120

View full text Add to dashboard Cite

We propose a novel video frame interpolation algorithm based on asymmetric bilateral motion estimation (ABME), which synthesizes an intermediate frame between two input frames. First, we predict symmetric bilateral motion fields to interpolate an anchor frame. Second, we estimate asymmetric bilateral motions fields from the anchor frame to the input frames. Third, we use the asymmetric fields to warp the input frames backward and reconstruct the intermediate frame. Last, to refine the intermediate frame, we develop a new synthesis network that generates a set of dynamic filters and a residual frame using local and global information. Experimental results show that the proposed algorithm achieves excellent performance on various datasets. The source codes and pretrained models are available at https://github.com/JunHeum/ABME.

show abstract

What Matters in Unsupervised Optical Flow

Cited by 128 publications

References 31 publications

Self-Supervised Monocular Scene Flow Estimation

Self-Supervised Monocular Scene Flow Estimation

Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses

BMBC: Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation

Contact Info

Product

Resources

About