Ergo, SMIRK is safe: a safety case for a machine learning component in a pedestrian automatic emergency brake system

Abstract

Integration of machine learning (ML) components in critical applications introduces novel challenges for software certification and verification. New safety standards and technical guidelines are under development to support the safety of ML-based systems, e.g., ISO 21448 SOTIF for the automotive domain and the Assurance of Machine Learning for use in Autonomous Systems (AMLAS) framework. SOTIF and AMLAS provide high-level guidance but the details must be chiseled out for each specific case. We initiated a research project with the goal to demonstrate a complete safety case for an ML component in an open automotive system. This paper reports results from an industry-academia collaboration on safety assurance of SMIRK, an ML-based pedestrian automatic emergency braking demonstrator running in an industry-grade simulator. We demonstrate an application of AMLAS on SMIRK for a minimalistic operational design domain, i.e., we share a complete safety case for its integrated ML-based component. Finally, we report lessons learned and provide both SMIRK and the safety case under an open-source license for the research community to reuse.

The Safety of Autonomy: A Systematic Approach

“…22 An example of the independent use of AMLAS (involving none of the AMLAS developers) was for the safety assurance of an emergency braking system for an AV, intended to protect pedestrians. 23 This can be seen as an example of assuring the SOTIF. These two examples are both embedded systems-the initial target of the framework-but AMLAS has also been successfully applied to decision support systems, for example, in healthcare.…”

“…25 While there have been some initial successes in applying the approach, there remain limitations and issues of maturity. As with the standards, our approach is not precise about the level of evidence needed for the safety casealthough both the definitions of SACE 11 and AMLAS 12 and the examples of the use of AMLAS 22,23,24 illustrate the approach and thus assist in interpreting and applying it. But this is work in progress, and there is more to be done.…”

Machine learning testing in an ADAS case study using simulation‐integrated bio‐inspired search‐based testing

“…However, how it shall be implemented and argued for being adequate and sufficient is highly up to the specific stakeholder organization. 31 In this regard, Hawkins et al 8 introduced a methodology for the assurance of ML in autonomous systems, called AMLAS. It presents a systematic process for integrating safety assurance into the development of ML systems and introduces verification procedures at different stages, for example, learning model verification and system-level (integration) verification that happens after integrating the ML model into the system.…”

“…Standards such as ISO 26262 Functional Safety 11 and ISO 21448 Safety of the Intended Functionality 10 often provide a high‐level view of the requirements that are required to be satisfied in a safety case for an ML‐driven system. However, how it shall be implemented and argued for being adequate and sufficient is highly up to the specific stakeholder organization 31 . In this regard, Hawkins et al 8 introduced a methodology for the assurance of ML in autonomous systems, called AMLAS.…”