Structuring Validation Targets of a Machine Learning Function Applied to Automated Driving

Gauerhof, Lydia; Munk, Peter; Burton, Simon

doi:10.1007/978-3-319-99130-6_4

Cited by 43 publications

(23 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section we apply the assurance case structure described above to the pedestrian recognition case study introduced in [10] and demonstrate how arguments regarding typical performance evaluation techniques can be strengthened or refuted. The performance requirements of the function used for the case study can be summarised as follows:…”

Section: Case Studymentioning

confidence: 99%

Confidence Arguments for Evidence of Performance in Machine Learning for Highly Automated Driving Functions

Burton

Gauerhof

Sethy

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Due to their ability to efficiently process unstructured and highly dimensional input data, machine learning algorithms are being applied to perception tasks for highly automated driving functions. The consequences of failures and insufficiencies in such algorithms are severe and a convincing assurance case that the algorithms meet certain safety requirements is therefore required. However, the task of demonstrating the performance of such algorithms is non-trivial, and as yet, no consensus has formed regarding an appropriate set of verification measures. This paper provides a framework for reasoning about the contribution of performance evidence to the assurance case for machine learning in an automated driving context and applies the evaluation criteria to a pedestrian recognition case study.

show abstract

Section: Case Studymentioning

confidence: 99%

Confidence Arguments for Evidence of Performance in Machine Learning for Highly Automated Driving Functions

Burton

Gauerhof

Sethy

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Of course, additional safety arguments are required that the set of assumptions is valid and maintained during operation, however we do not address this issue in more detail here. The specification of such properties (safety guarantees) has been explored by a number of authors [5,17,21,3] and can include the following categories:…”

Section: Related Workmentioning

confidence: 99%

A causal model of safety assurance for machine learning

Burton¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a framework based on a causal model of safety upon which effective safety assurance cases for ML-based applications can be built. In doing so, we build upon established principles of safety engineering as well as previous work on structuring assurance arguments for ML. The paper defines four categories of safety case evidence and a structured analysis approach within which these evidences can be effectively combined. Where appropriate, abstract formalisations of these contributions are used to illustrate the causalities they evaluate, their contributions to the safety argument and desirable properties of the evidences. Based on the proposed framework, progress in this area is re-evaluated and a set of future research directions proposed in order for tangible progress in this field to be made.

show abstract

“…To analyze the intended function reliability, the function boundaries and use cases need to be clarified. Consequently, defined use cases are only part of the intended function [7].…”

Section: Function Boundarymentioning

confidence: 99%

“…Other researchers suggested the utilization of vertical and horizontal pixel deviation between ground truth and detection to judge the confidence level [7]. In this case, the direction influences of bias can be considered, but it complicates the reliability expression.…”

Section: Statistical Reliability Of the Functionmentioning

confidence: 99%

Approach to Define the Reliability of Safety-Related Machine Learning Based Functions in Highly Automated Driving

Krini

Müller

et al. 2021

Proceedings of CECNet 2021

View full text Add to dashboard Cite

Current standards cannot cover the safety requirements of machine learning based functions used in highly automated driving. Because of the opacity of neural networks, some self-driving functions cannot be developed following the V-model. These functions require the expansion of the standards. This paper focuses on this gap and defines functional reliability for such functions to help the future standards control the quality of machine learning based functions. As an example, reliability functions for pedestrian detection are built. Since the quality criteria in computer vision do not consider safety, new approaches for expression and evaluation of this reliability are designed.

show abstract

Structuring Validation Targets of a Machine Learning Function Applied to Automated Driving

Cited by 43 publications

References 7 publications

Confidence Arguments for Evidence of Performance in Machine Learning for Highly Automated Driving Functions

Confidence Arguments for Evidence of Performance in Machine Learning for Highly Automated Driving Functions

A causal model of safety assurance for machine learning

Approach to Define the Reliability of Safety-Related Machine Learning Based Functions in Highly Automated Driving

Contact Info

Product

Resources

About