Trust in Automated Systems Test

Wojton, Heather; Porter, Daniel; Lane, Stephanie; Bieber, Chad; Madhavan, Poornima

doi:10.1037/t87674-000

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…New test methodologies, therefore, need to provide these DMAIL algorithms' assurance even as they continue to learn. In a report on operational testing of autonomy, the Institute for Defense Analysis (IDA) notes "a fundamental challenge in autonomy is developing trust in its decision-making capacity across all situations and environments it potentially will encounter" (Wojton et al 2019).…”

Section: A Intelligent Engineered Systems and The Challenges For Syst...mentioning

confidence: 99%

Explainable AI and Counterfactuals for Test and Evaluation of Intelligent Engineered Systems

Raz,

Miller,

Chang

et al. 2023

INCOSE International Symp

View full text Add to dashboard Cite

Systems Engineering (SE) based test and evaluation (T&E) approaches have proven crucial for successful realization of most modern‐day complex systems and system‐of‐systems solutions. The design, test, and evaluation of engineered systems with many technologically advanced and complex components hinges on well‐structured integration and life cycle evolution process models, clearly defined requirements, along with controllability, observability, and stability (COS) of components that transcend to the system‐level. However, integration of machine learning (ML), deep learning (DL) and artificially intelligent (AI) components invalidates some SE foundations as the DL algorithms are primarily data‐driven with opaque decision‐making constructs. As a result, the current SE T&E approaches, although necessary, have become insufficient to evaluate the adoption of ML/DL/AI methods in engineered systems. This paper proposes two new approaches—explainable AI (xAI) and counterfactual testing and evaluation (cT&E)—as addition to the SE tool set for T&E of intelligent engineered systems. An example of SE T&E considerations is provided based on a conceptual aircraft control system implementation comparing as a classical controller and a DL‐based reinforcement learning controller.

show abstract

Section: A Intelligent Engineered Systems and The Challenges For Syst...mentioning

confidence: 99%