Calibrating Uncertainty Models for Steering Angle Estimation

Hubschneider, Christian; Hutmacher, Robin; Zöllner, J. Marius

doi:10.1109/itsc.2019.8917207

Cited by 17 publications

(16 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, usually both types of uncertainty are captured [Lee et al, 2019b;Lee et al, 2019c;Lee et al, 2019a] by using the method proposed by , or by using DE, boostrap ensembles, or MDNs. The calibration plots presented in [Hubschneider et al, 2019] show that MCD has better out-ofthe-box calibration than bootstrap ensembles or MDNs; the last two methods are overconfident in their predictions. In this particular task, safety mechanisms have been proposed when uncertainty estimations surpass a given or learned threshold in order to improve vehicle safety [Michelmore et al, 2018;Michelmore et al, 2019;Lee et al, 2019b].…”

Section: Considerations Per Av Task Typementioning

confidence: 96%

“…Under ideal circumstances, we expect that the normalized outputs from a DNN (i.e softmax outputs) correspond to the true correctness likelihood [Guo et al, 2017]. From a frequentist perspective, this can be viewed as a discrepancy measure between local confidence (or uncertainty) predictions and the expected performance in the long-run [Hubschneider et al, 2019;Lakshminarayanan et al, 2017]. For example, we expect that a class predicted with probability p is correct p% of the time, i.e.…”

Section: Neural Network Calibrationmentioning

confidence: 99%

“…DDNs can be calibrated by using Temperature Scaling, a simple postprocessing technique [Guo et al, 2017], or more recently, Dirichelt calibration [Kull et al, 2019]. For a regression setting, [Kuleshov et al, 2018;Hubschneider et al, 2019] formalize the calibration notion for continuous variables, in which a p% confidence interval should contain the true outcome p% of the time.…”

Section: Neural Network Calibrationmentioning

confidence: 99%

“…-Steering Angle Prediction[Hubschneider et al, 2019] -Traffic Sign Recognition[Henne et al, 2020] -Semantic Segmentation[Gustafsson et al, 2019] -Depth Estimation[Gustafsson et al, 2019] …”

mentioning

confidence: 99%

See 3 more Smart Citations

A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications

Arnez,

Espinoza,

Radermacher

et al. 2020

Preprint

View full text Add to dashboard Cite

A key factor for ensuring safety in Autonomous Vehicles (AVs) is to avoid any abnormal behaviors under undesirable and unpredicted circumstances. As AVs increasingly rely on Deep Neural Networks (DNNs) to perform safety-critical tasks, different methods for uncertainty quantification have recently been proposed to measure the inevitable source of errors in data and models. However, uncertainty quantification in DNNs is still a challenging task. These methods require a higher computational load, a higher memory footprint, and introduce extra latency, which can be prohibitive in safety-critical applications. In this paper, we provide a brief and comparative survey of methods for uncertainty quantification in DNNs along with existing metrics to evaluate uncertainty predictions. We are particularly interested in understanding the advantages and downsides of each method for specific AV tasks and types of uncertainty sources.

show abstract

Section: Considerations Per Av Task Typementioning

confidence: 96%

Section: Neural Network Calibrationmentioning

confidence: 99%

Section: Neural Network Calibrationmentioning

confidence: 99%

“…-Steering Angle Prediction[Hubschneider et al, 2019] -Traffic Sign Recognition[Henne et al, 2020] -Semantic Segmentation[Gustafsson et al, 2019] -Depth Estimation[Gustafsson et al, 2019] …”

mentioning

confidence: 99%

See 2 more Smart Citations

A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications

Arnez,

Espinoza,

Radermacher

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Due to the promising role of uncertainty estimation in increasing autonomous and robotic systems' safety by indicating low confidence in output predictions -and consequently detecting failures -many authors in the field of robotic perception investigated and compared variations of the main methods of estimating uncertainty from DNNs. Examples include uncertainty estimation for steering angle estimation [43], road segmentation [44], visual odometry [45], and vehicle and object detection [46]- [48].…”

Section: ) Uncertainty Estimationmentioning

confidence: 99%

Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trends

2021

View full text Add to dashboard Cite

As deep learning continues to dominate all state-of-the-art computer vision tasks, it is increasingly becoming an essential building block for robotic perception. This raises important questions concerning the safety and reliability of learning-based perception systems. There is an established field that studies safety certification and convergence guarantees of complex software systems at design-time. However, the unknown future deployment environments of an autonomous system and the complexity of learning-based perception make the generalization of design-time verification to run-time problematic. In the face of this challenge, more attention is starting to focus on run-time monitoring of performance and reliability of perception systems with several trends emerging in the literature in the face of this challenge. This paper attempts to identify these trends and summarise the various approaches to the topic.

show abstract