RGGNet: Tolerance Aware LiDAR-Camera Online Calibration With Geometric Deep Learning and Generative Model

Yuan, Kaiwen; Guo, Zhenyu; Wang, Z. Jane

doi:10.1109/lra.2020.3026958

Cited by 68 publications

(48 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The above deep learning calibration methods ignore the tolerance within the error bounds. RGGNet [14] utilized the Riemannian geometry and deep generative model to build a tolerance-aware loss function.…”

Section: Deep Learning Approachmentioning

confidence: 99%

“…The initial calibration off-range ∆T was (±1.5 m, ±20 • ). To compare our method with other learning-based (CNN-based) methods, we utilized the same four test datasets [14] on the raw recordings of the KITTI dataset. Each test dataset was independent of the training dataset with the following test name configurations:…”

Section: Dataset Preparationmentioning

confidence: 99%

“…Besides, we applied another two se(3) error based evaluation metrics mentioned in [14]: mean error (MSEE) se(3) and mean re-calibration rate (MRR). MRR reflects how much noise us compensated by the re-calibration.…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…Thus, some efforts attempted to adopt deep learning to learn the 6-DoF rigid transformation between LiDAR and camera data. Deep learning-based extrinsic calibration methods [12][13][14][15] also demonstrate flexibility and adaptability in changing environments. Some methods [12,13] cannot meet the performance requirements of real-world applications.…”

Section: Introductionmentioning

confidence: 99%

“…Some methods [12,13] cannot meet the performance requirements of real-world applications. Some methods [14,15] regress the parameters directly, which introduces training difficulties. The gap between the performances of deep learning-based methods and the practical demands is being bridged.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

CFNet: LiDAR-Camera Registration Using Calibration Flow Network

Wang

Dou

et al. 2021

Sensors

View full text Add to dashboard Cite

As an essential procedure of data fusion, LiDAR-camera calibration is critical for autonomous vehicles and robot navigation. Most calibration methods require laborious manual work, complicated environmental settings, and specific calibration targets. The targetless methods are based on some complex optimization workflow, which is time-consuming and requires prior information. Convolutional neural networks (CNNs) can regress the six degrees of freedom (6-DOF) extrinsic parameters from raw LiDAR and image data. However, these CNN-based methods just learn the representations of the projected LiDAR and image and ignore the correspondences at different locations. The performances of these CNN-based methods are unsatisfactory and worse than those of non-CNN methods. In this paper, we propose a novel CNN-based LiDAR-camera extrinsic calibration algorithm named CFNet. We first decided that a correlation layer should be used to provide matching capabilities explicitly. Then, we innovatively defined calibration flow to illustrate the deviation of the initial projection from the ground truth. Instead of directly predicting the extrinsic parameters, we utilize CFNet to predict the calibration flow. The efficient Perspective-n-Point (EPnP) algorithm within the RANdom SAmple Consensus (RANSAC) scheme is applied to estimate the extrinsic parameters with 2D–3D correspondences constructed by the calibration flow. Due to its consideration of the geometric information, our proposed method performed better than the state-of-the-art CNN-based methods on the KITTI datasets. Furthermore, we also tested the flexibility of our approach on the KITTI360 datasets.

show abstract

Section: Deep Learning Approachmentioning

confidence: 99%

Section: Dataset Preparationmentioning

confidence: 99%

Section: Evaluation Metricsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

CFNet: LiDAR-Camera Registration Using Calibration Flow Network

Wang

Dou

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

Robust Environmental Perception of Multi-sensor Data Fusion

Pan,

Wang,

et al. 2023

Recent Advancements in Connected Autonomous Vehicle Technologies

View full text Add to dashboard Cite

Automatic targetless LiDAR–camera calibration: a survey

Xiao

Wang³

et al. 2022

Artif Intell Rev

View full text Add to dashboard Cite

The recent trend of fusing complementary data from LiDARs and cameras for more accurate perception has made the extrinsic calibration between the two sensors critically important. Indeed, to align the sensors spatially for proper data fusion, the calibration process usually involves estimating the extrinsic parameters between them. Traditional LiDAR-camera calibration methods often depend on explicit targets or human intervention, which can be prohibitively expensive and cumbersome. Recognizing these weaknesses, recent methods usually adopt the autonomic targetless calibration approach, which can be conducted at a much lower cost. This paper presents a thorough review of these automatic targetless LiDAR-camera calibration methods. Specifically, based on how the potential cues in the environment are retrieved and utilized in the calibration process, we divide the methods into four categories: information theory based, feature based, ego-motion based, and learning based methods. For each category, we provide an in-depth overview with insights we have gathered, hoping to serve as a potential guidance for researchers in the related fields.

show abstract

RGGNet: Tolerance Aware LiDAR-Camera Online Calibration With Geometric Deep Learning and Generative Model

Cited by 68 publications

References 35 publications

CFNet: LiDAR-Camera Registration Using Calibration Flow Network

CFNet: LiDAR-Camera Registration Using Calibration Flow Network

Robust Environmental Perception of Multi-sensor Data Fusion

Automatic targetless LiDAR–camera calibration: a survey

Contact Info

Product

Resources

About