A Mapping of Assurance Techniques for Learning Enabled Autonomous Systems to the Systems Engineering Lifecycle

Abstract

Learning enabled autonomous systems provide increased capabilities compared to traditional systems. However, the complexity of and probabilistic nature in the underlying methods enabling such capabilities present challenges for current systems engineering processes for assurance, and test, evaluation, verification, and validation (TEVV). This paper provides a preliminary attempt to map recently developed technical approaches in the assurance and TEVV of learning enabled autonomous systems (LEAS) literature to a traditional systems engineering v-model. This mapping categorizes such techniques into three main approaches: development, acquisition, and sustainment. We review the latest techniques to develop safe, reliable, and resilient learning enabled autonomous systems, without recommending radical and impractical changes to existing systems engineering processes. By performing this mapping, we seek to assist acquisition professionals by (i) informing comprehensive test and evaluation planning, and (ii) objectively communicating risk to leaders.

The National Airworthiness Council artificial intelligence working group (NACAIWG) summit proceedings 2022

“…For the 2022 summit, a sample of literature published since the 2021 summit was provided to introduce participants to updated work in the field of test and evaluation (T&E), CV, algorithmic assurance, and frameworks to provide assurance of algorithms. All papers were collected from the Purdue University Library 3 and Dissertations and Theses Database 4 and cited in references 5–14 …”