Focus on Defocus: Bridging the Synthetic to Real Domain Gap for Depth Estimation

Maximov, Maxim; Kevin, Galim,; Leal-Taixé, Laura

doi:10.1109/cvpr42600.2020.00115

Cited by 54 publications

(75 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also demonstrated its applicability using images captured in actual foggy scenes. For future work, we will extend the proposed method to depth-dependent degradation, other than light scattering, such as defocus blur [14,23].…”

Section: Discussionmentioning

confidence: 99%

Dehazing Cost Volume for Deep Multi-view Stereo in Scattering Media with Airlight and Scattering Coefficient Estimation

Fujimura

Sonogashira

Iiyama

2020

Preprint

View full text Add to dashboard Cite

We propose a learning-based multi-view stereo (MVS) method in scattering media, such as fog or smoke, with a novel cost volume, called the dehazing cost volume. Images captured in scattering media are degraded due to light scattering and attenuation caused by suspended particles. This degradation depends on scene depth; thus, it is difficult for traditional MVS methods to evaluate photometric consistency because the depth is unknown before threedimensional (3D) reconstruction. The dehazing cost volume can solve this chicken-and-egg problem of depth estimation and image restoration by computing the scattering effect using swept planes in the cost volume. We also propose a method of estimating scattering parameters, such as airlight, and a scattering coefficient, which are required for our dehazing cost volume. The output depth of a network with our dehazing cost volume can be regarded as a function of these parameters; thus, they are geometrically optimized with a sparse 3D point cloud obtained at a structurefrom-motion step. Experimental results on synthesized hazy images indicate the effectiveness of our dehazing cost volume against the ordinary cost volume regarding scattering media. We also demonstrated the applicability of our dehazing cost volume to real foggy scenes.

show abstract

Section: Discussionmentioning

confidence: 99%

Dehazing Cost Volume for Deep Multi-view Stereo in Scattering Media with Airlight and Scattering Coefficient Estimation

Fujimura

Sonogashira

Iiyama

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Defocus properties, also known as the "Bokeh effect", are well known since decades by photographers of art. They Guillaume Caron is with CNRS-AIST JRL (Joint Robotics Laboratory), IRL, AIST, Tsukuba, Japan and with Université de Picardie Jules Verne, MIS laboratory, Amiens, France guillaume.caron@u-picardie.fr have been exploited both in computer graphics for realistic rendering [14], [15], [16] and in computer vision for dense depth computation [17], [18], [19]. In short, the amount of defocus blur in the image depends on the depth difference between scene points and the plane in focus (which depth is set by the camera lens).…”

Section: B Related Workmentioning

confidence: 99%

Defocus-Based Direct Visual Servoing

Caron

2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Direct Visual Servoing (DVS) considers pixel brightness directly as input of robot control. Recent DVS variants consider image processing as smoothing or frequency domain transforms, resulting in large convergence domains.This paper proposes to consider defocus to optically smooth images without processing. The resulting Defocus-based DVS shows convergence domains competing with the state-of-theart, larger in some challenging cases, for lower complexity.

show abstract

“…To reduce the requirements of real-world images in depth estimation, [2][40] [41] explore image translation techniques to generate synthetic labeled data. [29] tackles synthetic to real depth estimation issue by using domain invariant defocus blur as direct supervision. [39] proposes a domain normalization approach of stereo matching that regularizes the distribution of learned representations to allow them to be invariant to domain differences.…”

Section: Domain Adaptationmentioning

confidence: 99%

Self-supervised Monocular Depth Estimation for All Day Images using Domain Separation

Liu¹,

Song²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Remarkable results have been achieved by DCNN based self-supervised depth estimation approaches. However, most of these approaches can only handle either day-time or night-time images, while their performance degrades for all-day images due to large domain shift and the variation of illumination between day and night images. To relieve these limitations, we propose a domain-separated network for self-supervised depth estimation of all-day images. Specifically, to relieve the negative influence of disturbing terms (illumination, etc.), we partition the information of day and night image pairs into two complementary sub-spaces: private and invariant domains, where the former contains the unique information (illumination, etc.) of day and night images and the latter contains essential shared information (texture, etc.). Meanwhile, to guarantee that the day and night images contain the same information, the domain-separated network takes the day-time images and corresponding night-time images (generated by GAN) as input, and the private and invariant feature extractors are learned by orthogonality and similarity loss, where the domain gap can be alleviated, thus better depth maps can be expected. Meanwhile, the reconstruction and photometric losses are utilized to estimate complementary information and depth maps effectively. Experimental results demonstrate that our approach achieves state-of-theart depth estimation results for all-day images on the challenging Oxford RobotCar dataset, proving the superiority of our proposed approach. Code and data split are available at https://github.com/LINA-lln/ADDS-DepthNet.

show abstract

Focus on Defocus: Bridging the Synthetic to Real Domain Gap for Depth Estimation

Cited by 54 publications

References 24 publications

Dehazing Cost Volume for Deep Multi-view Stereo in Scattering Media with Airlight and Scattering Coefficient Estimation

Dehazing Cost Volume for Deep Multi-view Stereo in Scattering Media with Airlight and Scattering Coefficient Estimation

Defocus-Based Direct Visual Servoing

Self-supervised Monocular Depth Estimation for All Day Images using Domain Separation

Contact Info

Product

Resources

About