Radar-Camera Pixel Depth Association for Depth Completion

Long, Y. F.; Morris, Daniel; Liu, Xiaoming; Castro, Marcos; Chakravarty, Punarjay; Narayanan, Praveen

doi:10.1109/cvpr46437.2021.01232

Cited by 51 publications

(42 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, the extended detections are sent to a two-stage architecture to output a denoised radar depth map. Long et al [116] proposed a probabilistic association method to model the uncertainties of radar detections. As shown in Figure 8c, radar points are transformed into a multi-channel enhanced radar (MER) image, with each channel representing the expanded radar depth at a specific confidence level of association.…”

Section: Depth Estimationmentioning

confidence: 99%

See 1 more Smart Citation

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

Zhou

Liu

Zhao

et al. 2022

Sensors

View full text Add to dashboard Cite

With recent developments, the performance of automotive radar has improved significantly. The next generation of 4D radar can achieve imaging capability in the form of high-resolution point clouds. In this context, we believe that the era of deep learning for radar perception has arrived. However, studies on radar deep learning are spread across different tasks, and a holistic overview is lacking. This review paper attempts to provide a big picture of the deep radar perception stack, including signal processing, datasets, labelling, data augmentation, and downstream tasks such as depth and velocity estimation, object detection, and sensor fusion. For these tasks, we focus on explaining how the network structure is adapted to radar domain knowledge. In particular, we summarise three overlooked challenges in deep radar perception, including multi-path effects, uncertainty problems, and adverse weather effects, and present some attempts to solve them.

show abstract

Section: Depth Estimationmentioning

confidence: 99%

“…As the ground truth, LiDAR has some inherent defects, such as sparsity, limited range, and holes with no reflections. Long et al [116] suggest to preprocess LiDAR points for better supervision. They accumulated multiple frames of LiDAR point clouds to improve density.…”

Section: Depth Estimationmentioning

confidence: 99%

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

Zhou

Liu

Zhao

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Lee et al [17] built upon the architecture from [10] with additional predictions for semantic segmentation and 2D object detection to improve the performance of depth estimation. Long et al [12] proposed a radar-to-pixel association stage as a first stage, and followed by a traditional depth completion method.…”

Section: Depth Estimation Using Camera and Radarmentioning

confidence: 99%

“…Although radar is a low cost and robust sensor, there are a few characteristics that bring disadvantages for radar as a depth guidance signal. As described in [10,11,12] in detail, the main disadvantages are sparseness, noisy measurements, and limited vertical field of view. For tackling sparseness and noise issues, Lin et al accumulated raw radar points from multiple frames and used prediction from first-stage to do filtering on raw noisy sparse radar [10].…”

Section: Radar Datamentioning

confidence: 99%

“…On the contrary, radar is known for its robustness and reliability against extreme weather and it is much cheaper than lidar sensors. Accordingly, several researchers start to fuse radar sensor data as a further guidance into camera-based depth estimation models [10,11,12,13,14] after the release of a large autonomous driving dataset nuScenes [15].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

How Much Depth Information can Radar Contribute to a Depth Estimation Model?

Lo¹,

Vandewalle²

2022

Preprint

View full text Add to dashboard Cite

Since the release of radar data in large scale autonomous driving dataset, many works have been proposed fusing radar data as an additional guidance signal into monocular depth estimation models. Although positive performances are reported, it is still hard to tell how much depth information radar can infer and contribute in depth estimation models. In this paper, we conduct two experiments to investigate the intrinsic depth capability of radar data using state-of-the-art depth estimation models. Our experiments demonstrate that the estimated depth from only sparse radar input can detect the shape of surroundings to a certain extent. Furthermore, the monocular depth estimation model supervised by preprocessed radar only during training can achieve 70% performance in δ 1 score compared to the baseline model trained with sparse lidar.

show abstract