Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review

Abstract

With the significant advancement of sensor and communication technology and the reliable application of obstacle detection techniques and algorithms, automated driving is becoming a pivotal technology that can revolutionize the future of transportation and mobility. Sensors are fundamental to the perception of vehicle surroundings in an automated driving system, and the use and performance of multiple integrated sensors can directly determine the safety and feasibility of automated driving vehicles. Sensor calibration is the foundation block of any autonomous system and its constituent sensors and must be performed correctly before sensor fusion and obstacle detection processes may be implemented. This paper evaluates the capabilities and the technical performance of sensors which are commonly employed in autonomous vehicles, primarily focusing on a large selection of vision cameras, LiDAR sensors, and radar sensors and the various conditions in which such sensors may operate in practice. We present an overview of the three primary categories of sensor calibration and review existing open-source calibration packages for multi-sensor calibration and their compatibility with numerous commercial sensors. We also summarize the three main approaches to sensor fusion and review current state-of-the-art multi-sensor fusion techniques and algorithms for object detection in autonomous driving applications. The current paper, therefore, provides an end-to-end review of the hardware and software methods required for sensor fusion object detection. We conclude by highlighting some of the challenges in the sensor fusion field and propose possible future research directions for automated driving systems.

Research on Design, Calibration and Real-Time Image Expansion Technology of Unmanned System Variable-Scale Panoramic Vision System

“…Due to the complexity of the camera projection model and the difficulty of assembling high-quality sensors, the accurate calibration of panoramic vision systems is more challenging than in ordinary vision systems [ 38 ]. Scale information is extracted from the environment, and panoramic systems calibration is the first necessary step to determine the mapping relationship between the environment point coordinates and image pixel coordinates, and to correct these errors in the application of panoramic vision systems.…”

A Literature Review of Performance Metrics of Automated Driving Systems for On-Road Vehicles

“…point cloud data from LiDARs) can also be used for object detection and tracking. A comparison of the three major sensors used for environment perception is provided in Table 1, which is compiled by reviewing multiple sources (Hasch et al, 2012;Murad et al, 2013;Patole et al, 2017;Campbell et al, 2018;Lin and Zhang, 2020;Lu et al, 2020;Wang et al, 2020;Zaarane et al, 2020;Yeong et al, 2021).…”

Automated Driving with Cooperative Perception Based on CVFH and Millimeter-Wave V2I Communications for Safe and Efficient Passing through Intersections

“…Moreover, sharing raw sensor data can contribute to liability problems in the case of accidents and improving the accuracy of object localization [ 20 ]. In [ 24 ], it discusses sharing raw and processed sensor data from the viewpoint of sensor fusion. High-level fusion, which shares the results of detection and tracking algorithm carried out by each sensor, can be realized under lower complexity and requires few communication resources.…”