“…Further research and extensive testing of autonomous systems are essential to assess all possible solutions to prevent malicious attacks and evaluate all possible sensors and system failure risks and alternative solutions in the case of sensors or system failures. A detailed discussion about the sensor fusion challenges, including adversarial attacks and possible preventions is beyond the scope of this paper (see [ 16 , 19 , 25 , 211 , 212 , 213 , 214 ] for a more comprehensive overview).…”
Section: Sensor Calibration and Sensor Fusion For Object Detectionmentioning
confidence: 99%
“…The sensing capabilities of an AV employing a diverse set of sensors is an essential element in the overall AD system; the cooperation and performance of these sensors can directly determine the viability and safety of an AV [ 16 ]. The selection of an appropriate array of sensors and their optimal configurations, which will, in essence be used to imitate the human ability to perceive and formulate a reliable picture of the environment, is one of the primary considerations in any AD system.…”
With the significant advancement of sensor and communication technology and the reliable application of obstacle detection techniques and algorithms, automated driving is becoming a pivotal technology that can revolutionize the future of transportation and mobility. Sensors are fundamental to the perception of vehicle surroundings in an automated driving system, and the use and performance of multiple integrated sensors can directly determine the safety and feasibility of automated driving vehicles. Sensor calibration is the foundation block of any autonomous system and its constituent sensors and must be performed correctly before sensor fusion and obstacle detection processes may be implemented. This paper evaluates the capabilities and the technical performance of sensors which are commonly employed in autonomous vehicles, primarily focusing on a large selection of vision cameras, LiDAR sensors, and radar sensors and the various conditions in which such sensors may operate in practice. We present an overview of the three primary categories of sensor calibration and review existing open-source calibration packages for multi-sensor calibration and their compatibility with numerous commercial sensors. We also summarize the three main approaches to sensor fusion and review current state-of-the-art multi-sensor fusion techniques and algorithms for object detection in autonomous driving applications. The current paper, therefore, provides an end-to-end review of the hardware and software methods required for sensor fusion object detection. We conclude by highlighting some of the challenges in the sensor fusion field and propose possible future research directions for automated driving systems.
“…Further research and extensive testing of autonomous systems are essential to assess all possible solutions to prevent malicious attacks and evaluate all possible sensors and system failure risks and alternative solutions in the case of sensors or system failures. A detailed discussion about the sensor fusion challenges, including adversarial attacks and possible preventions is beyond the scope of this paper (see [ 16 , 19 , 25 , 211 , 212 , 213 , 214 ] for a more comprehensive overview).…”
Section: Sensor Calibration and Sensor Fusion For Object Detectionmentioning
confidence: 99%
“…The sensing capabilities of an AV employing a diverse set of sensors is an essential element in the overall AD system; the cooperation and performance of these sensors can directly determine the viability and safety of an AV [ 16 ]. The selection of an appropriate array of sensors and their optimal configurations, which will, in essence be used to imitate the human ability to perceive and formulate a reliable picture of the environment, is one of the primary considerations in any AD system.…”
With the significant advancement of sensor and communication technology and the reliable application of obstacle detection techniques and algorithms, automated driving is becoming a pivotal technology that can revolutionize the future of transportation and mobility. Sensors are fundamental to the perception of vehicle surroundings in an automated driving system, and the use and performance of multiple integrated sensors can directly determine the safety and feasibility of automated driving vehicles. Sensor calibration is the foundation block of any autonomous system and its constituent sensors and must be performed correctly before sensor fusion and obstacle detection processes may be implemented. This paper evaluates the capabilities and the technical performance of sensors which are commonly employed in autonomous vehicles, primarily focusing on a large selection of vision cameras, LiDAR sensors, and radar sensors and the various conditions in which such sensors may operate in practice. We present an overview of the three primary categories of sensor calibration and review existing open-source calibration packages for multi-sensor calibration and their compatibility with numerous commercial sensors. We also summarize the three main approaches to sensor fusion and review current state-of-the-art multi-sensor fusion techniques and algorithms for object detection in autonomous driving applications. The current paper, therefore, provides an end-to-end review of the hardware and software methods required for sensor fusion object detection. We conclude by highlighting some of the challenges in the sensor fusion field and propose possible future research directions for automated driving systems.
“…Further research and extensive testing of autonomous systems are essential to assess all possible solutions to prevent malicious attacks and evaluate all possible sensors and system failure risks and; alternative solutions in the case of sensors or system failures. A detailed discussion about the sensor fusion challenges, including adversarial attacks and possible preventions is beyond the scope of this paper (see [16,202,[205][206][207][208][209] for a more comprehensive overview).…”
Section: Challenges Of Sensor Fusion For Safe and Reliable Environmenmentioning
confidence: 99%
“…The sensing capabilities of an AV employing a diverse set of sensors is an essential element in the overall AD system; the cooperation and performance of these sensors directly determines the viability and safety of an AV [13]. The selection of an appropriate array of sensors and their optimal configurations, which will, in essence be used to imitate the human ability to perceive and formulate a reliable picture of the environment, is one of the primary considerations in any AD system.…”
The market for autonomous vehicles (AV) is expected to experience significant growth over the coming decades and to revolutionize the future of transportation and mobility. The AV is a vehicle that is capable of perceiving its environment and perform driving tasks safely and efficiently with little or no human intervention and is anticipated to eventually replace conventional vehicles. Self-driving vehicles employ various sensors to sense and perceive their surroundings and, also rely on advances in 5G communication technology to achieve this objective. Sensors are fundamental to the perception of surroundings and the development of sensor technologies associated with AVs has advanced at a significant pace in recent years. Despite remarkable advancements, sensors can still fail to operate as required, due to for example, hardware defects, noise and environment conditions. Hence, it is not desirable to rely on a single sensor for any autonomous driving task. The practical approaches shown in recent research is to incorporate multiple, complementary sensors to overcome the shortcomings of individual sensors operating independently. This article reviews the technical performance and capabilities of sensors applicable to autonomous vehicles, mainly focusing on vision cameras, LiDAR and Radar sensors. The review also considers the compatibility of sensors with various software systems enabling the multi-sensor fusion approach for obstacle detection. This review article concludes by highlighting some of the challenges and possible future research directions.
“…Important progress has been realized, particularly in autonomous driving research, since its inception in 1980 and the DARPA urban competition in 2007 [ 1 , 2 ]. However, up to now, developing a reliable autonomous driving system remained a challenge [ 3 ]. Object detection and recognition are some of the challenging tasks involved in achieving accurate, robust, reliable, and real-time perceptions [ 4 ].…”
The progress brought by the deep learning technology over the last decade has inspired many research domains, such as radar signal processing, speech and audio recognition, etc., to apply it to their respective problems. Most of the prominent deep learning models exploit data representations acquired with either Lidar or camera sensors, leaving automotive radars rarely used. This is despite the vital potential of radars in adverse weather conditions, as well as their ability to simultaneously measure an object’s range and radial velocity seamlessly. As radar signals have not been exploited very much so far, there is a lack of available benchmark data. However, recently, there has been a lot of interest in applying radar data as input to various deep learning algorithms, as more datasets are being provided. To this end, this paper presents a survey of various deep learning approaches processing radar signals to accomplish some significant tasks in an autonomous driving application, such as detection and classification. We have itemized the review based on different radar signal representations, as it is one of the critical aspects while using radar data with deep learning models. Furthermore, we give an extensive review of the recent deep learning-based multi-sensor fusion models exploiting radar signals and camera images for object detection tasks. We then provide a summary of the available datasets containing radar data. Finally, we discuss the gaps and important innovations in the reviewed papers and highlight some possible future research prospects.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.