Artificial agents’ explainability to support trust: considerations on timing and context

Abstract

Strategies for improving the explainability of artificial agents are a key approach to support the understandability of artificial agents’ decision-making processes and their trustworthiness. However, since explanations are not inclined to standardization, finding solutions that fit the algorithmic-based decision-making processes of artificial agents poses a compelling challenge. This paper addresses the concept of trust in relation to complementary aspects that play a role in interpersonal and human–agent relationships, such as users’ confidence and their perception of artificial agents’ reliability. Particularly, this paper focuses on non-expert users’ perspectives, since users with little technical knowledge are likely to benefit the most from “post-hoc”, everyday explanations. Drawing upon the explainable AI and social sciences literature, this paper investigates how artificial agent’s explainability and trust are interrelated at different stages of an interaction. Specifically, the possibility of implementing explainability as a trust building, trust maintenance and restoration strategy is investigated. To this extent, the paper identifies and discusses the intrinsic limits and fundamental features of explanations, such as structural qualities and communication strategies. Accordingly, this paper contributes to the debate by providing recommendations on how to maximize the effectiveness of explanations for supporting non-expert users’ understanding and trust.

Addressing the Black Box of AI—A Model and Research Agenda on the Co-constitution of Aging and Artificial Intelligence

“…However, research in the field of Socio-gerontechnology ( Peine et al, 2021 ) has highlighted that older adults—including people with high care needs—routinely and actively engage with technological innovations ( Gallistl et al, 2021 ), underscoring the importance of older adult’s agentic engagement and subjectivity in research on AI. Recent work on explainability of AI has proposed to focus on sense-making practices as a crucial element in understanding how people perceive and understand AI ( Papagni et al, 2023 ). There is significant room in the context of aging for work on explainability, as well as expanding participation in AI development and governance.…”

Trust Development and Explainability: A Longitudinal Study with a Personalized Assistive System

“…Furthermore, recent long term studies [26,27] have found time to be an important factor influencing trust in repeated interactions with such systems. Hence, the dynamic nature of trust necessitates studying its relationship with (and effects of) performance/accuracy and explainability at different moments of an interaction [28][29][30][31].…”

“…Importantly, compared to the other trust restoration strategies, explainability comes with one major advantage. In this regard, studies suggest that perceiving an anomaly in a system's behavior represents the main trigger for an explanation request [65,66] and that providing explanations for such anomalies (and mistakes), may result in increased trust and reliance by shedding light on the causes of the anomalous behavior rather than just offering a re-structuring of the relationship [31,39,67]. At the same time, high levels of trust in a faulty system may still be dangerous, even if the system can explain its mistakes.…”

Human Centricity in the Relationship Between Explainability and Trust in AI

“…Some people are prone to having overly high expectations of technology, a tendency referred to as automation bias, or they may underestimate the risks associated with the actions of artificial agents. On the other hand, lack of trust in AI and skepticism toward technology are also widely observed attitudes [8].…”