A GPU -accelerated Deep Stereo- LiDAR Fusion for Real-time High-precision Dense Depth Sensing

Zhong, Chonghao; Gu, Jianfeng; Chen, Gang

doi:10.23919/date51398.2021.9474009

Cited by 4 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike the existing fusion strategies, VPN directly embeds the point cloud into the cost volume, which can disseminate effective information to the nearby voxels in the cost volume and reduce uncertainty (Choe et al, 2021 ). FastFusion was proposed as a binary neural network integrating stereo-matching information as input (Meng et al, 2021 ). Meng et al fused stereo-matching data and sparse point cloud data-based LiDAR aggregation.…”

Section: Related Workmentioning

confidence: 99%

Real-time depth completion based on LiDAR-stereo for autonomous driving

et al. 2023

View full text Add to dashboard Cite

The integration of multiple sensors is a crucial and emerging trend in the development of autonomous driving technology. The depth image obtained by stereo matching of the binocular camera is easily influenced by environment and distance. The point cloud of LiDAR has strong penetrability. However, it is much sparser than binocular images. LiDAR-stereo fusion can neutralize the advantages of the two sensors and maximize the acquisition of reliable three-dimensional information to improve the safety of automatic driving. Cross-sensor fusion is a key issue in the development of autonomous driving technology. This study proposed a real-time LiDAR-stereo depth completion network without 3D convolution to fuse point clouds and binocular images using injection guidance. At the same time, a kernel-connected spatial propagation network was utilized to refine the depth. The output of dense 3D information is more accurate for autonomous driving. Experimental results on the KITTI dataset showed that our method used real-time techniques effectively. Further, we demonstrated our solution's ability to address sensor defects and challenging environmental conditions using the p-KITTI dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Real-time depth completion based on LiDAR-stereo for autonomous driving

et al. 2023

View full text Add to dashboard Cite

show abstract

“…This manuscript extends its preliminary conference version (Meng & Meng, 2021) with the following major differences: (1) Since the depth estimation of Meng & Meng (2021) is less smooth and appearance coordination with an image texture, we reformulate the refinement process in the stereo‐matching domain by integrating sparse LiDAR measurement to enhance estimation accuracy. (2) We demonstrate a viable training pipeline that imparts the refinement network with pseudo ground truth and novel perceptual loss function to handle the inaccurate disparity matching.…”

Section: Introductionmentioning

confidence: 99%

“…Moreover, in the generalization experiment which trains models on the KITTI (Menze & Geiger, 2015) data set and evaluates them on the DrivingStereo (Yang et al, 2019) data set, our proposed fusion framework achieves both the best accuracy of depth estimation and the fastest inference speed on unseen scenarios with a notable lead compared with other fusion algorithms, revealing strong practical usability in the real-world applications. This manuscript extends its preliminary conference version (Meng & Meng, 2021) with the following major differences:…”

mentioning

confidence: 97%

FastFusion: Deep stereo‐LiDAR fusion for real‐time high‐precision dense depth sensing

Meng

Li²,

Zhong

et al. 2023

Journal of Field Robotics

Self Cite

View full text Add to dashboard Cite

Light detection and ranging (LiDAR) and stereo cameras are two generally used solutions for perceiving 3D information. The complementary properties of these two sensor modalities motivate a fusion to derive practicable depth sensing toward real‐world applications. Promoted by deep neural network (DNN) techniques, recent works achieve superior performance on accuracy. However, the complex architecture and the sheer number of DNN parameters often lead to poor generalization capacity and non‐real‐time computing. In this paper, we present FastFusion, a three‐stage stereo‐LiDAR deep fusion scheme, which integrates the LiDAR priors into each step of classical stereo‐matching taxonomy, gaining high‐precision dense depth sensing in a real‐time manner. We integrate stereo‐LiDAR information by taking advantage of a compact binary neural network and utilize the proposed cross‐based LiDAR trust aggregation to further fuse the sparse LiDAR measurements in the back‐end of stereo matching. To align the photometrical of the input image and the depth of the estimation, we introduce a refinement network to guarantee consistency. More importantly, we present a graphic processing unit‐based acceleration framework for providing a low‐latency implementation of FastFusion, gaining both accuracy improvement and real‐time responsiveness. In the experiments, we demonstrate the effectiveness and practicability of FastFusion, which obtains a significant speedup over state‐of‐the‐art baselines while achieving comparable accuracy on depth sensing. The video demo for real‐time depth estimation of FastFusion on the real‐world driving scenario is available at https://youtu.be/nP7cls2BA8s.

show abstract

LiDAR-SGM: Semi-Global Matching on LiDAR Point Clouds and Their Cost-Based Fusion into Stereo Matching

Forkel¹,

Wuensche²

2023

2023 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

A GPU -accelerated Deep Stereo- LiDAR Fusion for Real-time High-precision Dense Depth Sensing

Cited by 4 publications

References 13 publications

Real-time depth completion based on LiDAR-stereo for autonomous driving

Real-time depth completion based on LiDAR-stereo for autonomous driving

FastFusion: Deep stereo‐LiDAR fusion for real‐time high‐precision dense depth sensing

LiDAR-SGM: Semi-Global Matching on LiDAR Point Clouds and Their Cost-Based Fusion into Stereo Matching

Contact Info

Product

Resources

About