Calibration-Net:LiDAR and Camera Auto-Calibration using Cost Volume and Convolutional Neural Network

Duy, An Nguyen; Yoo, Myungsik

doi:10.1109/icaiic54071.2022.9722671

Cited by 7 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, CalibRCNN [24] calculated 6-DOF transformation between 3D LiDAR and 2D camera in real-time by utilizing the LSTM network to extract features and managed both geometric loss and photometric loss to refine the calibration accuracy. Recently, Duy and Yoo [25] proposed Calibration-Net, which provides autocalibration between LiDAR and camera using cost volume and Convolutional Neural Network, suggesting the possibility of extrinsic calibration between range sensors using depth information.…”

Section: Learning-based Sensor Calibrationmentioning

confidence: 99%

Imaging radar and LiDAR image translation for 3-DOF extrinsic calibration

Jung,

Jang,

Jung

et al. 2024

Intel Serv Robotics

View full text Add to dashboard Cite

The integration of sensor data is crucial in the field of robotics to take full advantage of the various sensors employed. One critical aspect of this integration is determining the extrinsic calibration parameters, such as the relative transformation, between each sensor. The use of data fusion between complementary sensors, such as radar and LiDAR, can provide significant benefits, particularly in harsh environments where accurate depth data is required. However, noise included in radar sensor data can make the estimation of extrinsic calibration challenging. To address this issue, we present a novel framework for the extrinsic calibration of radar and LiDAR sensors, utilizing CycleGAN as a method of image-to-image translation. Our proposed method employs translating radar bird-eye-view images into LiDAR-style images to estimate the 3-DOF extrinsic parameters. The use of image registration techniques, as well as deskewing based on sensor odometry and B-spline interpolation, is employed to address the rolling shutter effect commonly present in spinning sensors. Our method demonstrates a notable improvement in extrinsic calibration compared to filter-based methods using the MulRan dataset.

show abstract

Section: Learning-based Sensor Calibrationmentioning

confidence: 99%

Imaging radar and LiDAR image translation for 3-DOF extrinsic calibration

Jung,

Jang,

Jung

et al. 2024

Intel Serv Robotics

View full text Add to dashboard Cite

show abstract

“…Deep learning networks complete one or several tasks within them. Generally speaking, in order to maintain the consistency of extracted features from different data branches, a data conversion (data preparation [ 39 , 40 , 41 , 42 , 43 ]) is usually performed before data input. The basic process of an AEPE method is shown in Figure 4 .…”

Section: Deep Learning-based Extrinsic Calibrationmentioning

confidence: 99%

“…Slightly different from previous models, they added a spatial pyramid pooling (SPP) layer [ 67 ] in the feature matching module to generate fixed size feature maps and to achieve the input with any size. The Calibration-Net [ 42 ] constructed by Duy et al applied the LCCNet [ 46 ] model to calibrate with two depth map branches. And Zhang et al [ 68 ] used RegNet as the backbone network, but improved it with geometric and photometric losses.…”

Section: Deep Learning-based Extrinsic Calibrationmentioning

confidence: 99%

“…In some scenarios and studies [ 14 , 22 , 70 ], it is necessary to calibrate binocular or multiple cameras with LiDAR. The commonly used method is to convert the multi-camera image and LiDAR point cloud into disparity images [ 71 ] or depth maps [ 42 , 66 ] (which are essentially the same, and can be converted to each other through Equation (4)), thereby making the subsequent feature extraction and matching of network models more accurate. The multi-camera estimation of depth information has always been a key technology in the field of machine vision, with many mature methods including local feature-based methods [ 72 , 73 ], global feature-based methods [ 74 ], CNN model-based methods [ 75 ], etc.…”

Section: Deep Learning-based Extrinsic Calibrationmentioning

confidence: 99%

“…Sensors 2024, 24, 3878 2 of 42 Although there have been some reviews on LiDAR and camera calibration, such as Nie et al [1], Wang et al [2], Li et al [4], Liu et al [16], Khurana et al [17], which all contain content about learning-based methods, they only provide a relatively brief summary. With the rapid development of this technology, we believe that it is necessary and important to conduct a comprehensive and in-depth review on this topic.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Review of Deep Learning-Based LiDAR and Camera Extrinsic Calibration

Tan,

Zhang,

Teng

et al. 2024

Sensors

View full text Add to dashboard Cite

Extrinsic parameter calibration is the foundation and prerequisite for LiDAR and camera data fusion of the autonomous system. This technology is widely used in fields such as autonomous driving, mobile robots, intelligent surveillance, and visual measurement. The learning-based method is one of the targetless calibrating methods in LiDAR and camera calibration. Due to its advantages of fast speed, high accuracy, and robustness under complex conditions, it has gradually been applied in practice from a simple theoretical model in just a few years, becoming an indispensable and important method. This paper systematically summarizes the research and development of this type of method in recent years. According to the principle of calibration parameter estimation, learning-based calibration algorithms are divided into two categories: accurate calibrating estimation and relative calibrating prediction. The evolution routes and algorithm frameworks of these two types of algorithms are elaborated, and the methods used in the algorithms’ steps are summarized. The algorithm mechanism, advantages, limitations, and applicable scenarios are discussed. Finally, we make a summary, pointing out existing research issues and trends for future development.

show abstract

Enhancing the accuracy of prototype learning in road anomaly segmentation by adding adversarial perturbations to data

Lin,

Lin

2023

Multimed Tools Appl

View full text Add to dashboard Cite

Calibration-Net:LiDAR and Camera Auto-Calibration using Cost Volume and Convolutional Neural Network

Cited by 7 publications

References 11 publications

Imaging radar and LiDAR image translation for 3-DOF extrinsic calibration

Imaging radar and LiDAR image translation for 3-DOF extrinsic calibration

A Review of Deep Learning-Based LiDAR and Camera Extrinsic Calibration

Enhancing the accuracy of prototype learning in road anomaly segmentation by adding adversarial perturbations to data

Contact Info

Product

Resources

About