A Heuristic Based Framework for Improving Design of Unmanned Systems by Quantifying and Assessing Operator Trust

Abstract

Operator trust in autonomous capability is a critical factor affecting the success of fielded unmanned systems. However, current means of identifying trusted system behavior do not sufficiently inform guidelines for designing an engineering solution. We present a novel Trust Assessment Framework for providing actionable and formative feedback for building an unmanned system that promotes operators’ initial adoption of and reliance on its autonomous capabilities. Based on a review of the unmanned systems literature and a series of inquiries focusing on operators of unmanned aerial vehicles, we have identified a set of Heuristics for Trusted Autonomy and behaviorally anchored Trust Assessment Scales that enable quantitative feedback for each heuristic category. The Heuristics and Scales were incorporated into a Trust Assessment Process consisting of a Heuristic Evaluation and subsequent scoring. This trust assessment methodology can be applied directly to assessing autonomous aerial systems. Extensions to other platforms and domains are being investigated.

Trusted Partners: Human-Machine Teaming and the Future of Military AI

“…Human trust as it pertains to technology, and more specifically, automated, autonomous and/or AI-enabled systems, can be organized in three categories: dispositional, situational, and learned. 26 Dispositional trust refers to a human's inherent tendency to trust automation, which varies based on a multitude of factors such as a person's age, personality, or culture. 27 For example, a global survey of 18,000 adults aged 16-64 found that younger generations trust AI more than older generations.…”

Human Factors Considerations for Enabling Functional Use of Exosystems in Operational Environments

“…With an understanding of where and for what aspect of the situation (e.g., exosystem or environment) breakdowns may occur in the evaluation, future system design requirements can be developed. Trust can also be evaluated with a set of questions through selfreported or observer-based measures [92]- [95]. These heuristics were developed for unmanned systems and human-computer interactions scenarios and should be adapted by the exosystem community with an agreed-upon terminology such that the questions and actionable items are aligned with relevant exosystem taxonomies and use cases.…”