Spatial Correspondence With Generative Adversarial Network: Learning Depth From Monocular Videos

Wu, Zhenyao; Wu, Xinyi; Zhang, Xiaoping; Wang, Song; Ju, Lili

doi:10.1109/iccv.2019.00759

Cited by 24 publications

(13 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their network learns from the ground truth labels in synthetic domain as well as the epipolar geometry of the real domain, thereby achieving competitive results. Wu et al [105] improve the architecture of the generator by utilizing a spatial correspondence module for feature matching and an attention mechanism for feature re-weighting.…”

Section: Semi-supervised Monocular Depth Estimationmentioning

confidence: 99%

Monocular depth estimation based on deep learning: An overview

Zhao¹,

Sun²,

Zhang³

et al. 2020

Sci. China Technol. Sci.

203

View full text Add to dashboard Cite

Depth information is important for autonomous systems to perceive environments and estimate their own state. Traditional depth estimation methods, like structure from motion and stereo vision matching, are built on feature correspondences of multiple viewpoints. Meanwhile, the predicted depth maps are sparse. Inferring depth information from a single image (monocular depth estimation) is an ill-posed problem. With the rapid development of deep neural networks, monocular depth estimation based on deep learning has been widely studied recently and achieved promising performance in accuracy. Meanwhile, dense depth maps are estimated from single images by deep neural networks in an end-to-end manner. In order to improve the accuracy of depth estimation, different kinds of network frameworks, loss functions and training strategies are proposed subsequently. Therefore, we survey the current monocular depth estimation methods based on deep learning in this review. Initially, we conclude several widely used datasets and evaluation indicators in deep learning-based depth estimation. Furthermore, we review some representative existing methods according to different training manners: supervised, unsupervised and semi-supervised. Finally, we discuss the challenges and provide some ideas for future researches in monocular depth estimation.

show abstract

Section: Semi-supervised Monocular Depth Estimationmentioning

confidence: 99%

Monocular depth estimation based on deep learning: An overview

Zhao¹,

Sun²,

Zhang³

et al. 2020

Sci. China Technol. Sci.

203

View full text Add to dashboard Cite

show abstract

Section: Monocular Depth Estimationmentioning

confidence: 99%

“…Other work concentrates on multi-view stereo (MVS), which operates on unordered image sets [45,40,42,33,37,14,63,62,61]. Not requiring the ground truth depth and camera poses during training, self-supervised MVS methods [42,33,37,14,63,62,61] leverage cost volumes to process sequences of frames at test time. Compared with the base method of MVS, these methods can predict the depth using images captured by moving cameras and do not need camera poses during training time.…”

Section: Monocular Depth Estimationmentioning

confidence: 99%

DurLAR: A High-Fidelity 128-Channel LiDAR Dataset with Panoramic Ambient and Reflectivity Imagery for Multi-Modal Autonomous Driving Applications

Ismail

Shum

et al. 2021

2021 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

We present DurLAR, a high-fidelity 128-channel 3D Li-DAR dataset with panoramic ambient (near infrared) and reflectivity imagery, as well as a sample benchmark task using depth estimation for autonomous driving applications. Our driving platform is equipped with a high resolution 128 channel LiDAR, a 2MPix stereo camera, a lux meter and a GNSS/INS system. Ambient and reflectivity images are made available along with the LiDAR point clouds to facilitate multi-modal use of concurrent ambient and reflectivity scene information. Leveraging DurLAR, with a resolution exceeding that of prior benchmarks, we consider the task of monocular depth estimation and use this increased availability of higher resolution, yet sparse ground truth scene depth information to propose a novel joint supervised/selfsupervised loss formulation. We compare performance over both our new DurLAR dataset, the established KITTI benchmark and the Cityscapes dataset. Our evaluation shows our joint use supervised and self-supervised loss terms, enabled via the superior ground truth resolution and availability within DurLAR improves the quantitative and qualitative performance of leading contemporary monocular depth estimation approaches (RM SE = 3.639, SqRel = 0.936).

show abstract

“…p interp (r) = max MonoDis [9] AM3D [26] GS3D [10] ROI-10D [6] 3D [41], [42], it is significantly more difficult for approaches that are solely built upon monocular RGB input as predicting depth from monocular images is a challenging task [43], [44]. Furthermore, Average Orientation Similarity (AOS) is used to assess the ability of 3D-GCK to correctly predict the object orientation [32].…”

Section: Metricsmentioning

confidence: 99%

Single-Shot 3D Detection of Vehicles from Monocular RGB Images via Geometry Constrained Keypoints in Real-Time

Gählert,

Wan,

Jourdan

et al. 2020

Preprint

View full text Add to dashboard Cite

In this paper we propose a novel 3D single-shot object detection method for detecting vehicles in monocular RGB images. Our approach lifts 2D detections to 3D space by predicting additional regression and classification parameters and hence keeping the runtime close to pure 2D object detection. The additional parameters are transformed to 3D bounding box keypoints within the network under geometric constraints. Our proposed method features a full 3D description including all three angles of rotation without supervision by any labeled ground truth data for the object's orientation, as it focuses on certain keypoints within the image plane. While our approach can be combined with any modern object detection framework with only little computational overhead, we exemplify the extension of SSD for the prediction of 3D bounding boxes. We test our approach on different datasets for autonomous driving and evaluate it using the challenging KITTI 3D Object Detection as well as the novel nuScenes Object Detection benchmarks. While we achieve competitive results on both benchmarks we outperform current state-of-the-art methods in terms of speed with more than 20 FPS for all tested datasets and image resolutions.

show abstract

Spatial Correspondence With Generative Adversarial Network: Learning Depth From Monocular Videos

Cited by 24 publications

References 37 publications

Monocular depth estimation based on deep learning: An overview

Monocular depth estimation based on deep learning: An overview

DurLAR: A High-Fidelity 128-Channel LiDAR Dataset with Panoramic Ambient and Reflectivity Imagery for Multi-Modal Autonomous Driving Applications

Single-Shot 3D Detection of Vehicles from Monocular RGB Images via Geometry Constrained Keypoints in Real-Time

Contact Info

Product

Resources

About