Structured Adversarial Training for Unsupervised Monocular Depth Estimation

Mehta, Ishit; Sakurikar, Parikshit; Narayanan, P. J.

doi:10.1109/3dv.2018.00044

Cited by 39 publications

(17 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[16] showed that the inclusion of a local structure based appearance loss [65] significantly improved depth estimation performance compared to simple pairwise pixel differences [67,13,76]. [29] extended this approach to include an error fitting term, and [44] explored combining it with an adversarial based loss to encourage realistic looking synthesized images. Finally, inspired by [72], [73] use ground truth depth to train an appearance matching term.…”

Section: Appearance Based Lossesmentioning

confidence: 99%

Digging Into Self-Supervised Monocular Depth Estimation

Godard

Aodha

Firman³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

1,893

2,752

View full text Add to dashboard Cite

Per-pixel ground-truth depth data is challenging to acquire at scale. To overcome this limitation, self-supervised learning has emerged as a promising alternative for training models to perform monocular depth estimation. In this paper, we propose a set of improvements, which together result in both quantitatively and qualitatively improved depth maps compared to competing self-supervised methods.Research on self-supervised monocular training usually explores increasingly complex architectures, loss functions, and image formation models, all of which have recently helped to close the gap with fully-supervised methods. We show that a surprisingly simple model, and associated design choices, lead to superior predictions. In particular, we propose (i) a minimum reprojection loss, designed to robustly handle occlusions, (ii) a full-resolution multi-scale sampling method that reduces visual artifacts, and (iii) an auto-masking loss to ignore training pixels that violate camera motion assumptions. We demonstrate the effectiveness of each component in isolation, and show high quality, state-of-the-art results on the KITTI benchmark.

show abstract

Section: Appearance Based Lossesmentioning

confidence: 99%

Digging Into Self-Supervised Monocular Depth Estimation

Godard

Aodha

Firman³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

1,893

2,752

View full text Add to dashboard Cite

show abstract

“…With Synthetic Depth: Synthetic data is an interesting source of ground-truth depths and/or stereo pairs. Instead of the usual photometric loss, domain adaptation is possible using generative adversarial networks [25], or by leveraging the ability of stereo matching networks to better generalize to real world data [11]. Luo et al [21] demonstrate how synthetic data can be incorporated into single-image depth estimation with a two stage process.…”

Section: Additional Supervisionmentioning

confidence: 99%

“…This loss is usually the distance between a reference image and the depth-guided reprojection of other views into that reference viewpoint. Depth regression is optimized and relative poses come from stereo camera calibration in a training-from-stereo setting [9,7,25,27,26], while depth values and camera poses can be optimized jointly when training on videos [42,20,37,22,38,32,44,36,28].…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Monocular Depth Hints

Watson¹,

Firman²,

Brostow

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

223

178

View full text Add to dashboard Cite

Monocular depth estimators can be trained with various forms of self-supervision from binocular-stereo data to circumvent the need for high-quality laser scans or other ground-truth data. The disadvantage, however, is that the photometric reprojection losses used with selfsupervised learning typically have multiple local minima. These plausible-looking alternatives to ground truth can restrict what a regression network learns, causing it to predict depth maps of limited quality. As one prominent example, depth discontinuities around thin structures are often incorrectly estimated by current state-of-the-art methods.Here, we study the problem of ambiguous reprojections in depth prediction from stereo-based self-supervision, and introduce Depth Hints to alleviate their effects. Depth Hints are complementary depth suggestions obtained from simple off-the-shelf stereo algorithms. These hints enhance an existing photometric loss function, and are used to guide a network to learn better weights. They require no additional data, and are assumed to be right only sometimes. We show that using our Depth Hints gives a substantial boost when training several leading self-supervised-from-stereo models, not just our own. Further, combined with other good practices, we produce state-of-the-art depth predictions on the KITTI benchmark. We demonstrate that our selective training using DepthHints is a general enhancement that can improve multiple leading self-supervised training algorithms, allowing our implementations to reach better minima. The Depth Hints can come from the same stereo image data, via, e.g. OpenCV's stereo estimates [13,14].3. We show that our selective training with Depth Hints, coupled with sensible network design choices, leads us to outperform most other algorithms. We achieve state-of-the-art results on the KITTI dataset [8], outperforming both our baseline model and previously published results.

show abstract

“…Departing from conventional CNN, new technologies are being introduced. Adversarial learning of Generative Adversarial Network (GAN) is applied to train MDE models [25], [26]. Depth and pose prediction models are used as a generator to synthesize a target image, and a discriminator distinguishes real images from synthesized images.…”

Section: Related Workmentioning

confidence: 99%

Self-Supervised Monocular Depth Estimation With Extensive Pretraining

Choi¹

2021

IEEE Access

View full text Add to dashboard Cite

Structured Adversarial Training for Unsupervised Monocular Depth Estimation

Cited by 39 publications

References 21 publications

Digging Into Self-Supervised Monocular Depth Estimation

Digging Into Self-Supervised Monocular Depth Estimation

Self-Supervised Monocular Depth Hints

Self-Supervised Monocular Depth Estimation With Extensive Pretraining

Contact Info

Product

Resources

About