Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision

Kumar, Varun; Milz, Stefan; Witt, Christian; Simón, Martín; Amende, Karl; Petzold, Johannes; Yogamani, Senthil; Pech, Timo

doi:10.1109/itsc.2018.8569665

Cited by 43 publications

(26 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, Menze et al [25] running time is 50 minutes per frame which makes it impossible for usage in a real-time application such as the autonomous driving. Deep learning algorithms are becoming successful beyond object detection [30] for applications like visual SLAM [26], depth estimation [17], soiling detection [33] but it is still relatively less explored for MOD task.…”

Section: Related Workmentioning

confidence: 99%

FuseMODNet: Real-Time Camera and LiDAR Based Moving Object Detection for Robust Low-Light Autonomous Driving

Rashed

Ramzy

Vaquero

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

View full text Add to dashboard Cite

Moving object detection is a critical task for autonomous vehicles. As dynamic objects represent higher collision risk than static ones, our own ego-trajectories have to be planned attending to the future states of the moving elements of the scene. Motion can be perceived using temporal information such as optical flow. Conventional optical flow computation is based on camera sensors only, which makes it prone to failure in conditions with low illumination. On the other hand, LiDAR sensors are independent of illumination, as they measure the time-of-flight of their own emitted lasers. In this work, we propose a robust and real-time CNN architecture for Moving Object Detection (MOD) under low-light conditions by capturing motion information from both camera and LiDAR sensors. We demonstrate the impact of our algorithm on KITTI dataset where we simulate a low-light environment creating a novel dataset "Dark-KITTI". We obtain a 10.1% relative improvement on Dark-KITTI, and a 4.25% improvement on standard KITTI relative to our baselines. The proposed algorithm runs at 18 fps on a standard desktop GPU using 256 × 1224 resolution images.

show abstract

Section: Related Workmentioning

confidence: 99%

FuseMODNet: Real-Time Camera and LiDAR Based Moving Object Detection for Robust Low-Light Autonomous Driving

Rashed

Ramzy

Vaquero

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

View full text Add to dashboard Cite

show abstract

“…dynamic pixels. Depth task provides scaleaware distance in 3D space validated by occlusion corrected LiDAR depth [16]. The model is trained jointly using the public WoodScape [17] dataset comprising 8k samples and evaluated on 2k samples.…”

Section: Supervised Trainingmentioning

confidence: 99%

Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving

Sobh¹,

Hamed²,

Kumar³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Deep neural networks (DNNs) have accomplished impressive success in various applications, including autonomous driving perception tasks, in recent years. On the other hand, current deep neural networks are easily fooled by adversarial attacks. This vulnerability raises significant concerns, particularly in safety-critical applications. As a result, research into attacking and defending DNNs has gained much coverage. In this work, detailed adversarial attacks are applied on a diverse multi-task visual perception deep network across distance estimation, semantic segmentation, motion detection, and object detection. The experiments consider both white and black box attacks for targeted and un-targeted cases, while attacking a task and inspecting the effect on all the others, in addition to inspecting the effect of applying a simple defense method. We conclude this paper by comparing and discussing the experimental results, proposing insights and future work. The visualizations of the attacks are available at https://youtu.be/R3JUV41aiPY.

show abstract

“…They typically estimate a dense depth map from raw LiDAR measurements. Current deeplearning based methods for depth completion (Xu et al, 2019;Tang et al, 2020;Jaritz et al, 2018;Park et al, 2020;Ma and Karaman, 2018;Kumar et al, 2018;Zhao et al, 2021) usually learn to regress ground-truth depth maps in a fully-supervised setup (Figure 1b). Such approaches generally operate over visual and LiDAR inputs.…”

Section: Background and Related Workmentioning

confidence: 99%

LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR

Bartoccioni¹,

Zablocki²,

Pérez³

et al. 2021

Preprint

View full text Add to dashboard Cite

Vision-based depth estimation is a key feature in autonomous systems, which often relies on a single camera or several independent ones. In such a monocular setup, dense depth is obtained with either additional input from one or several expensive LiDARs, e.g., with 64 beams, or camera-only methods, which suffer from scale-ambiguity and infinite-depth problems. In this paper, we propose a new alternative of densely estimating metric depth by combining a monocular camera with a light-weight LiDAR, e.g., with 4 beams, typical of today's automotive-grade mass-produced laser scanners. Inspired by recent self-supervised methods, we introduce a novel framework, called LiDARTouch, to estimate dense depth maps from monocular images with the help of "touches" of LiDAR, i.e., without the need for dense ground-truth depth. In our setup, the minimal LiDAR input contributes on three different levels: as an additional model's input, in a self-supervised LiDAR reconstruction objective function, and to estimate changes of pose (a key component of self-supervised depth estimation architectures). Our LiDARTouch framework achieves new state of the art in self-supervised depth estimation on the KITTI dataset, thus supporting our choices of integrating the very sparse LiDAR signal with other visual features. Moreover, we show that the use of a few-beam LiDAR alleviates scale ambiguity and infinite-depth issues that camera-only methods suffer from. We also demonstrate that methods from the fully-supervised depth-completion literature can be adapted to a self-supervised regime with a minimal LiDAR signal.

show abstract

Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision

Cited by 43 publications

References 25 publications

FuseMODNet: Real-Time Camera and LiDAR Based Moving Object Detection for Robust Low-Light Autonomous Driving

FuseMODNet: Real-Time Camera and LiDAR Based Moving Object Detection for Robust Low-Light Autonomous Driving

Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving

LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR

Contact Info

Product

Resources

About