Joint Calibration of a Multimodal Sensor System for Autonomous Vehicles

Abstract

Multimodal sensor systems require precise calibration if they are to be used in the field. Due to the difficulty of obtaining the corresponding features from different modalities, the calibration of such systems is an open problem. We present a systematic approach for calibrating a set of cameras with different modalities (RGB, thermal, polarization, and dual-spectrum near infrared) with regard to a LiDAR sensor using a planar calibration target. Firstly, a method for calibrating a single camera with regard to the LiDAR sensor is proposed. The method is usable with any modality, as long as the calibration pattern is detected. A methodology for establishing a parallax-aware pixel mapping between different camera modalities is then presented. Such a mapping can then be used to transfer annotations, features, and results between highly differing camera modalities to facilitate feature extraction and deep detection and segmentation methods.

Dynamic Validation of Calibration Accuracy and Structural Robustness of a Multi-Sensor Mobile Robot

“…A substantial corpus of research has been conducted on the topic of sensor calibration. These include camera-to-camera calibration [6], camera-to-LiDAR calibration [7], camerato-IMU calibration [8], and calibration between different sensors and vehicle bodies [9]. Calibration serves to elucidate the relative relationships between sensors, and the absolute relationships between certain sensors and the vehicle body are clarified, which is of paramount importance for subsequent applications of multi-sensor fusion.…”