Introspective Failure Prediction for Autonomous Driving Using Late Fusion of State and Camera Information

Kuhn, Christopher B.; Hofbauer, Markus; Petrović, Goran; Steinbach, Eckehard

doi:10.1109/tits.2020.3044813

Cited by 12 publications

(5 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is a way to identify multiple items only by giving the whole picture to CNN without raster scanning the image. YOLO (You Only Look Once) is a representative method [9] in which an object rectangle and an object group is given for each local region split by a 7 × 7 grid. First, feature maps are formed through convolution and pooling of raw photos.…”

Section: B Application Of Cnn To Object Detection Taskmentioning

confidence: 99%

Advancing Autonomous Navigation: Deep Learning Techniques for Self-Driving Cars

Kapadia,

Mehta

2023

IJRASET

View full text Add to dashboard Cite

The fast development of self-driving car technology has brought forth radical breakthroughs in the world of transportation and movement. This study paper looks into the paradigm shift allowed by deep learning methods in getting stable and reliable autonomous steering for self-driving cars. Leveraging the power of artificial intelligence, particularly deep learning models, this study presents a comprehensive exploration of the multifaceted aspects involved in developing a self-driving car system that can adeptly perceive its environment, make intelligent decisions, and ensure passenger and pedestrian safety. The paper first reviews the basic concepts of self-driving cars, getting into the challenges associated with real-time awareness, scene understanding, and decision-making in complex and dynamic settings. It then shows the important role that deep learning plays in handling these challenges by studying various neural network designs designed for different tasks, including picture recognition, object detection, semantic segmentation, and path planning. The study encompasses convolutional neural networks (CNNs) for picture analysis.

show abstract

Section: B Application Of Cnn To Object Detection Taskmentioning

confidence: 99%

Advancing Autonomous Navigation: Deep Learning Techniques for Self-Driving Cars

Kapadia,

Mehta

2023

IJRASET

View full text Add to dashboard Cite

show abstract

“…Instead of restricting the system, our work aims to refine the system in order to allow it to handle as many scenarios as possible. In works such as [21], [22] or [23], future disengagements of an autonomous system were predicted by monitoring the input and output of the car. In this work, we show that external factors can also be used to predict system performance using a straightforward decision tree classifier.…”

Section: Related Workmentioning

confidence: 99%

Situation-Aware Model Refinement for Semantic Image Segmentation

Habermayr

Hofbauer

Zacchi³

et al. 2021

2021 IEEE International Intelligent Transportation Systems Conference (ITSC)

Self Cite

View full text Add to dashboard Cite

The quality of semantic image segmentation models can be affected by external factors such as weather or daytime. Those factors can lead to safety-critical mistakes. In this work, we propose a systematic approach to detect and alleviate such weaknesses of semantic segmentation models. We systematically evaluate a semantic segmentation model under different external factors and analyze which factors have the largest impact on the performance. Then, we collect new training data under the most harmful external factors and finetune the model. We use the CARLA simulator to obtain driving data under various environment settings. We deploy a state-ofthe-art semantic segmentation model in two distinct driving environments. Then, we use the proposed process to detect which external factors affect model performance the most. We collect new training data under those factors and fine-tune the model. The proposed approach outperforms collecting the same amount of random additional data by up to 10.6%. Our results show the benefit of using an iterative refinement approach as opposed to merely collecting larger data sets. Finally, we use the knowledge about which factors affect performance the most to train a simple decision tree classifier to predict the model's performance given the current external factors. Problematic environments can be detected at an average accuracy of 87.5%.

show abstract

“…Sensor fusion involves the close knitted integration and processing of data from multiple sensors for a more comprehensive and accurate understanding of the environment. − It is a crucial aspect of autonomous driving technology. , A variety of sensors, including light detection and ranging (LiDAR), millimeter-wave radar (Radar), and cameras, are extensively utilized to capture diverse information about the vehicle’s surroundings. − However, individual sensors are easily subject to various interferences such as changing weather conditions, electromagnetic disturbances, laser obstruction etc., significantly affecting the measurement accuracy and reliability of the entire system. , Late fusion in autonomous driving involves processing data from various sensors independently and merging their outputs at a later stage . It is particularly useful in addressing the aforementioned challenges and is expected to revolutionize future autonomous driving technology.…”

Section: Introductionmentioning

confidence: 99%

Integration of MoS₂ Memtransistor Devices and Analogue Circuits for Sensor Fusion in Autonomous Vehicle Target Localization

Tan,

Guo,

et al. 2024

ACS Nano

View full text Add to dashboard Cite

In contemporary autonomous driving systems relying on sensor fusion, traditional digital processors encounter challenges associated with analogue-to-digital conversion and iterative vector−matrix operations, which are encumbered by limitations in terms of response time and energy consumption. In this study, we present an analogue Kalman filter circuit based on molybdenum disulfide (MoS 2 ) memtransistor, designed to accelerate sensor fusion for precise localization in autonomous vehicle applications. The nonvolatile memory characteristics of the memtransistor allow for the storage of a fixed Kalman gain, which eliminates the data convergence and thus accelerates the processing speeds. Additionally, the modulation of multiple conductance states by the gate terminal enables fast adaptability to diverse autonomous driving scenarios by tuning multiple Kalman filter gains. Our proposed analogue Kalman filter circuit accurately estimates the position coordinates of target vehicles by fusing sensor data from light detection and ranging (LiDAR), millimeter-wave radar (Radar), and camera, and it successfully solves real-word problems in a signal-free crossroad intersection. Notably, our system achieves a 1000-fold improvement in energy efficiency compared to that of digital circuits. This work underscores the viability of a memtransistor for achieving fast, energy-efficient real-time sensing, and continuous signal processing in advanced sensor fusion technology.

show abstract

Introspective Failure Prediction for Autonomous Driving Using Late Fusion of State and Camera Information

Cited by 12 publications

References 34 publications

Advancing Autonomous Navigation: Deep Learning Techniques for Self-Driving Cars

Advancing Autonomous Navigation: Deep Learning Techniques for Self-Driving Cars

Situation-Aware Model Refinement for Semantic Image Segmentation

Integration of MoS₂ Memtransistor Devices and Analogue Circuits for Sensor Fusion in Autonomous Vehicle Target Localization

Contact Info

Product

Resources

About

Introspective Failure Prediction for Autonomous Driving Using Late Fusion of State and Camera Information

Cited by 12 publications

References 34 publications

Advancing Autonomous Navigation: Deep Learning Techniques for Self-Driving Cars

Advancing Autonomous Navigation: Deep Learning Techniques for Self-Driving Cars

Situation-Aware Model Refinement for Semantic Image Segmentation

Integration of MoS2 Memtransistor Devices and Analogue Circuits for Sensor Fusion in Autonomous Vehicle Target Localization

Contact Info

Product

Resources

About

Integration of MoS₂ Memtransistor Devices and Analogue Circuits for Sensor Fusion in Autonomous Vehicle Target Localization