Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual Information

Pandey, Gaurav; McBride, James R.; Savarese, Silvio; Eustice, Ryan M.

doi:10.1002/rob.21542

Cited by 236 publications

(232 citation statements)

References 46 publications

(66 reference statements)

Supporting

Mentioning

208

Contrasting

Unclassified

Order By: Relevance

“…More recently, the idea has been transferred to robotics for calibration of visual cameras to LIDAR scanners [15], [16]. This sensor registration has mostly been considered an offline task due to the expense of generating synthetic views for calibration.…”

Section: Related Workmentioning

confidence: 99%

Visual localization within LIDAR maps for automated urban driving

Wolcott

Eustice

2014

2014 IEEE/RSJ International Conference on Intelligent Robots and Systems

Self Cite

317

147

View full text Add to dashboard Cite

Abstract-This paper reports on the problem of map-based visual localization in urban environments for autonomous vehicles. Self-driving cars have become a reality on roadways and are going to be a consumer product in the near future. One of the most significant road-blocks to autonomous vehicles is the prohibitive cost of the sensor suites necessary for localization. The most common sensor on these platforms, a three-dimensional (3D) light detection and ranging (LIDAR) scanner, generates dense point clouds with measures of surface reflectivity-which other state-of-the-art localization methods have shown are capable of centimeter-level accuracy. Alternatively, we seek to obtain comparable localization accuracy with significantly cheaper, commodity cameras. We propose to localize a single monocular camera within a 3D prior groundmap, generated by a survey vehicle equipped with 3D LIDAR scanners. To do so, we exploit a graphics processing unit to generate several synthetic views of our belief environment. We then seek to maximize the normalized mutual information between our real camera measurements and these synthetic views. Results are shown for two different datasets, a 3.0 km and a 1.5 km trajectory, where we also compare against the state-of-the-art in LIDAR map-based localization.

show abstract

Section: Related Workmentioning

confidence: 99%

Visual localization within LIDAR maps for automated urban driving

Wolcott

Eustice

2014

2014 IEEE/RSJ International Conference on Intelligent Robots and Systems

Self Cite

317

147

View full text Add to dashboard Cite

show abstract

“…ToF and lidar) have been provided [10,75,82]. The range error statistics of these sensors are nearly independent and homogeneous [36], which allow point set registration (PSR) methods to utilize approximate joint PDFs of the transformed point clouds.…”

Section: Cramér-rao Bound Of Structured-light Sensorsmentioning

confidence: 99%

Optimal 6D Object Pose Estimation with Commodity Depth Sensors

Landau

View full text Add to dashboard Cite

Accurate 6D object pose estimation, as well as other model-based shape matching objectives such as object detection, classification, and shape inspection, is prominent and necessary in a large number of domains and applications. This includes household and robotics applications to automate various tasks, where input depth measurements are aligned to a corresponding model of the object, such as a CAD that is composed of several interconnected parts, depending on the model's level of detail and complexity. Industrial metrology systems also compare the shapes of engineered components and equipment to their corresponding 3D models for quality assessment and anomaly detection; this often requires very costly and high resolution projection and detection mechanisms. Structuredlight laser scanners have, however, emerged as a viable option to provide cheap and reasonably accurate depth estimates, which have recently become commodity-priced within this decade due to the maturity of the technology. These sensors are also robust for semi-transparent and reflective surfaces, dynamic scenery, ambient background light, and temperature drift. Certain structured-light light coding methods unfortunately need to implement a complicated, non-invertible transformation of a distorted light pattern to generate the pixel-by-pixel depth estimates, thereby leading to a loss of information. This can result in poorly estimated or even missing depth data, especially when the object has small 3D features or non-ideal surface properties. Moreover, the informative rigid body constraint is blurred when the nonlinear 2D depth image to 3D pseudo-measurement point cloud transformation is applied, which is the domain where most model-based shape matching methods operate.Alternatively, estimating pose from information early in the structured-light sensing/processing chain has the potential to alleviate errors induced in subsequent nonlinear processing steps that are in fact locally scene dependent. This motivates one of the main contributions from this dissertation, which presents an asymptotically optimal maximum likelihood estimation method that operates directly on the raw output IR images. This is made possible by an extensive study on a commonly utilized commodity-priced structured-light sensor, which contributed to the proposed high-fidelity IR and depth image predictor and simulator that models the physics of the transmit and receive optics,

show abstract

“…In contrast, we propose to use scene reflectivity to generate virtual views, which enables the use of highly accurate calibration targets. Pandey et al [21] also do not require a calibration target. The authors propose a calibration via minimizing the mutual information between the camera pixels and the laser scanner reflectivity information.…”

Section: Related Workmentioning

confidence: 99%

Accurate laser scanner to camera calibration with application to range sensor evaluation

Fuersattel

Plank

Maier

et al. 2017

IPSJ T Comput Vis Appl

View full text Add to dashboard Cite

Multi-modal sensory data plays an important role in many computer vision and robotics tasks. One popular multi-modal pair is cameras and laser scanners. To overlay and jointly use the data from both modalities, it is necessary to calibrate the sensors, i.e., to obtain the spatial relation between the sensors. Computing such a calibration is challenging as both sensors provide quite different data: cameras yield color or brightness information, laser scanners yield 3-D points. However, several laser scanners additionally provide reflectances, which turn out to make calibration to a camera well feasible. To this end, we first estimate a rough alignment of the coordinate systems of both modalities. Then, we use the laser scanner reflectances to compute a virtual image of the scene. Stereo calibration on the virtual image and the camera image are then used to compute a refined, high-accuracy calibration. It is encouraging that the accuracies in our experiments are comparable to camera-camera stereo setups and outperform another of other target-based calibration approach. This shows that the proposed algorithm reliably integrates the point cloud with the intensity image. As an example application, we use the calibration results to obtain ground-truth distance images for range cameras. Furthermore, we utilize this data to investigate the accuracy of the Microsoft Kinect V2 time-of-flight and the Intel RealSense R200 structured light camera.

show abstract

Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual Information

Cited by 236 publications

References 46 publications

Visual localization within LIDAR maps for automated urban driving

Visual localization within LIDAR maps for automated urban driving

Optimal 6D Object Pose Estimation with Commodity Depth Sensors

Accurate laser scanner to camera calibration with application to range sensor evaluation

Contact Info

Product

Resources

About