From ‘automation’ to ‘autonomy’: the importance of trust repair in human–machine interaction

Abstract

Modern interactions with technology are increasingly moving away from simple human use of computers as tools to the establishment of human relationships with autonomous entities that carry out actions on our behalf. In a recent commentary, Peter Hancock issued a stark warning to the field of human factors that attention must be focused on the appropriate design of a new class of technology: highly autonomous systems. In this article, we heed the warning and propose a human-centred approach directly aimed at ensuring that future human-autonomy interactions remain focused on the user's needs and preferences. By adapting literature from industrial psychology, we propose a framework to infuse a unique human-like ability, building and actively repairing trust, into autonomous systems. We conclude by proposing a model to guide the design of future autonomy and a research agenda to explore current challenges in repairing trust between humans and autonomous systems. Practitioner Summary: This paper is a call to practitioners to re-cast our connection to technology as akin to a relationship between two humans rather than between a human and their tools. To that end, designing autonomy with trust repair abilities will ensure future technology maintains and repairs relationships with their human partners.

Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces

“…Future work should concentrate on how to best present uncertainty information, for instance by unobtrusive means that do not increase operator workload and require fewer additional glances (Kunze et al, 2017). Acknowledging the uncertainty of automated agents may aid the shift from viewing automation as a tool to collaborative automation (de Visser, Pak, & Shaw, 2018…”

Taxonomy of Trust-Relevant Failures and Mitigation Strategies

“…In ongoing studies, several actions of trust repair have been proposed, including apologies, promises, internal or external attribution, and the showing of consistent series of trustworthy actions [4,14]. In an emergency setting, where an apology right after violated trust has not recovered trust, an apology right before the next trust decision point has repaired trust.…”

“Dave...I can assure you ...that it’s going to be all right ...” A Definition, Case for, and Survey of Algorithmic Assurances in Human-Autonomy Trust Relationships

“…de Visser et al [30] also speci cally discusses di erent methods by which AIAs can be more human-like in order to 'repair trust' with users (here, trust repair is roughly analogous to assurances, but focuses on re-building trust a er it is lost). Among several other possibilities, they suggest that an AIA might repair trust by anthropomorphizing (responding using a human communication channel), or by explaining its actions in the same way a person would.…”