Toward Certification of Machine-Learning Systems for Low Criticality Airborne Applications

Dmitriev, Konstantin; Schumann, Johann; Holzapfel, Florian

doi:10.1109/dasc52595.2021.9594467

Cited by 9 publications

(6 citation statements)

References 21 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The module alerts the human in case an unsafe action is detected. • Dmitriev et al [14]: work aimed at showing that current airborne certification standards can be complied with, in the case of a low-criticality ML-based system, if certain assumptions about the development workflow are applied. Nevertheless, these assumptions are only applicable to supervised learning systems.…”

Section: Current Regulatory Frameworkmentioning

confidence: 99%

“…Consequently, it is still unclear how this research can reach end users, although there is a lot of interest in making this happen, given their potential. Recent work highlights the incompatibilities between the conventional design assurance approach and aspects of ML systems [14,15]. In early 2023, the European Union Aviation Safety Agency (EASA) published the concept paper: 'First usable guidance for Level 1&2 machine learning applications' [16], which aims at providing guidelines for applicants introducing ML to safety-related or environment-related applications in the aviation domain.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Certification of Reinforcement Learning Applications for Air Transport Operations Based on Criticality and Autonomy

Ribeiro,

Tseremoglou,

Santos

2024

AIAA SCITECH 2024 Forum

View full text Add to dashboard Cite

Despite its success in various research domains, Reinforcement Learning (RL) faces challenges in its application to air transport operations due to the rigorous certification standards of the aviation industry. The existing regulatory framework fails to provide adequate, acceptable means of compliance for RL applications, and thus, there is no legal framework for their safe deployment yet. Guidelines must be formulated to certify RL models aimed at air transport operations to enable real-world utilisation of these promising methods. These guidelines must consider the unique characteristics of these models, deviating from the methodology of current guidelines crafted before the emergence of ML applications. The paper proposes novel certification requirements for RL models based on their technical characteristics, safety-criticality, and autonomy. This framework covers the choice of the RL algorithm and analyses the actions, agents, environment, and potential hazards and risks of the RL application. Additionally, this work outlines the evidence the certification applicant must present to demonstrate compliance with these requirements. While this framework is not a complete solution for the complex problem of certifying RL, it is intended to serve as an initial framework which can be extended in cooperation with regulatory entities.

show abstract

Section: Current Regulatory Frameworkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Certification of Reinforcement Learning Applications for Air Transport Operations Based on Criticality and Autonomy

Ribeiro,

Tseremoglou,

Santos

2024

AIAA SCITECH 2024 Forum

View full text Add to dashboard Cite

show abstract

“…Research, specifically regarding the certification of ML in relation to existing traditional standards, such as DO-178C, has also been done recently [10][11][12].…”

Section: Related Work a ML Safety Assurancementioning

confidence: 99%

Ensuring Safety of Machine Learning Components Using Operational Design Domain

Torens

Juenger

Schirmer

et al. 2023

AIAA SCITECH 2023 Forum

View full text Add to dashboard Cite

The introduction of machine learning in the aviation domain is an ongoing process. This is also true for safety-critical domains, especially for the area of Urban Air Mobility. A significant growth in number of air taxis and an increasing level of autonomy is to be expected allowing for operating a large number of air taxis in complex urban environments. Due to the complexity of the tasks and the environment, key autonomy functions will be realized using machine learning, for example the camera-based detection of objects. However, the safety assurance for avionics systems using machine learning components is challenging. This work investigates safety and verification aspects of machine learning components. A camera-based detection of humans on the ground, e.g. to assess a potential landing area, serves as an example for an machine learning-based autonomy functio and was integrated into an Unmanned Aircraft. In the context of this exemplary machine learning component, the concept of Operational Design Domain as recently adapted European Aviation SafetyAgency in the context of machine learning assurance is described along with other key concepts of machine learning assurance. Furthermore, runtime assurance is used to monitor conformance to the Operational Design Domain during flight. The presented flight test results indicate that monitoring the Operational Design Domain can support performance as well as the safety of the operation.

show abstract

“…For the 2022 summit, a sample of literature published since the 2021 summit was provided to introduce participants to updated work in the field of test and evaluation (T&E), CV, algorithmic assurance, and frameworks to provide assurance of algorithms. All papers were collected from the Purdue University Library 3 and Dissertations and Theses Database 4 and cited in references 5–14 …”

Section: Brief Overview Of Applicable Academic Literaturementioning

confidence: 99%

The National Airworthiness Council artificial intelligence working group (NACAIWG) summit proceedings 2022

Parry

Costello

Rupert³

et al. 2023

Systems Engineering

View full text Add to dashboard Cite

During the 2021 summit, a baseline overview of literature pertaining to the field of computer vision (CV) was provided. For the 2022 summit, a sample of literature published since the 2021 summit was provided to introduce participants to updated work in the field of test and evaluation (T&E), CV, algorithmic assurance, and frameworks to provide assurance of algorithms. All papers were collected from the Purdue University Library 3 and Dissertations and Theses Database 4 and cited in references. 5-14 Applicable standards governing LECBuilding off the standards survey conducted for the 2021 summit, a follow-on survey of standards for all modes of transportation (e.g., rail, sea, road, etc.) was conducted in support of the 2022 summit to determine if any notable improvements or advancements had occurred.

show abstract

Toward Certification of Machine-Learning Systems for Low Criticality Airborne Applications

Cited by 9 publications

References 21 publications

Certification of Reinforcement Learning Applications for Air Transport Operations Based on Criticality and Autonomy

Certification of Reinforcement Learning Applications for Air Transport Operations Based on Criticality and Autonomy

Ensuring Safety of Machine Learning Components Using Operational Design Domain

The National Airworthiness Council artificial intelligence working group (NACAIWG) summit proceedings 2022

Contact Info

Product

Resources

About