Time Synchronization and Space Registration of Roadside LiDAR and Camera

Abstract

The sensing system consisting of Light Detection and Ranging (LiDAR) and a camera provides complementary information about the surrounding environment. To take full advantage of multi-source data provided by different sensors, an accurate fusion of multi-source sensor information is needed. Time synchronization and space registration are the key technologies that affect the fusion accuracy of multi-source sensors. Due to the difference in data acquisition frequency and deviation in startup time between LiDAR and the camera, asynchronous data acquisition between LiDAR and camera is easy to occur, which has a significant influence on subsequent data fusion. Therefore, a time synchronization method of multi-source sensors based on frequency self-matching is developed in this paper. Without changing the sensor frequency, the sensor data are processed to obtain the same number of data frames and set the same ID number, so that the LiDAR and camera data correspond one by one. Finally, data frames are merged into new data packets to realize time synchronization between LiDAR and camera. Based on time synchronization, to achieve spatial synchronization, a nonlinear optimization algorithm of joint calibration parameters is used, which can effectively reduce the reprojection error in the process of sensor spatial registration. The accuracy of the proposed time synchronization method is 99.86% and the space registration accuracy is 99.79%, which is better than the calibration method of the Matlab calibration toolbox.

Robust Calibration Technique for Precise Transformation of Low-Resolution 2D LiDAR Points to Camera Image Pixels in Intelligent Autonomous Driving Systems

“…The image that exactly represents the same scene point should be acquired within 33 milliseconds of the obtained 2D LiDAR scan. As mentioned in [24], the present work assumed that the time taken to capture one LiDAR scan is T L and for capturing camera images, it is T C . For 'm' LiDAR scans, the captured image frames are n T L T C…”

A Practical Large-Scale Roadside Multi-View Multi-Sensor Spatial Synchronization Framework for Intelligent Transportation Systems

“…For instance, In the method proposed in [5], a set of sensors, including one LiDAR and two cameras, undergoes internal calibration followed by an external calibration with another set of sensors, leading to significant accumulated errors. Similarly, approaches in [3], [6] employ Zhang's method [7] for camera calibration. Du et al [4], initially compute the homography matrix based on four corresponding points for camera calibration, followed by a spatio-temporal synchronization optimization model.…”

“…Currently, there is a substantial research gap in the synchronization of multiple views and sensors in large-scale roadside deployments. Most existing studies focuses on experimental setups, as exemplified by Zheng et al's work [5], which primarily explores controlled scenarios [6], or single-view scenarios as discussed by Du et al [4]. Roadside spatial synchronization typically involves the CST, as shown in Fig.…”

“…Notably, before joint calibration, the camera and radar must be calibrated separately in advance. These methods [4]- [6] assume that sensor calibration has been manually conducted, yielding either accurate or preliminary results.…”