Nudging through Friction: An Approach for Calibrating Trust in Explainable AI

Abstract

Explainability has become an essential requirement for safe and effective collaborative Human-AI environments, especially when generating recommendations through black-box modality. One goal of eXplainable AI (XAI) is to help humans calibrate their trust while working with intelligent systems, i.e., avoid situations where human decision-makers over-trust the AI when it is incorrect, or under-trust the AI when it is correct. XAI, in this context, aims to help humans understand AI reasoning and decide whether to follow or reject its recommendations. However, recent studies showed that users, on average, continue to overtrust (or under-trust) AI recommendations which is an indication of XAI's failure to support trust calibration. Such a failure to aid trust calibration was due to the assumption that XAI users would cognitively engage with explanations and interpret them without bias. In this work, we hypothesize that XAI interaction design can play a role in helping users' cognitive engagement with XAI and consequently enhance trust calibration. To this end, we propose friction as a Nudge-based approach to help XAI users to calibrate their trust in AI and present the results of a preliminary study of its potential in fulfilling that role.

How the different explanation classes impact trust calibration: The case of clinical decision support systems

“…For instance, P9 asked to customise the number of data features in Local explanations, "The average pharmacist does not need to see all these factors that the AI is considering, some of them are just simple rules". Our observations are aligned with recent studies that showed that long and redundant explanations made participants skip them (Naiseh et al, 2021c) and decreased participants' satisfaction with the explanation (Narayanan et al, 2018). In summary, explanations easiness of use and their modalities could be critical to engaging users with them when participants' time constraints and the difficulty of the task are primary issues.…”

“…One approach to successful trust calibration is eXplainable AI (XAI) which refers to an AI component that explains AI recommendations to humans receiving them (Naiseh et al, 2021c). Explainability has been identified as a requirement to promote reliability and trust in the AI output and also to ensure humans remain in control (Holzinger, 2021).…”

“…While many studies emphasised the requisite of explainability to support trust calibration (Lai and Tan, 2019;Naiseh et al, 2021c;Zhang et al, 2020), several recent studies found little evidence that the XAI class has a significant impact on trust calibration (Jacobs et al, 2021). Many reasons could have contributed to this effect.…”

“…In fact, several studies suggest that XAI class output (Wang et al, 2019), its level of transparency (Kulesza et al, 2013;Guesmi et al, 2021) and framing (Narayanan et al, 2018) are all factors that can affect both humans' cognition and affect. Another reason for the effect of XAI class on trust calibration is the nature of humans' cognitive biases (Naiseh et al, 2021c). For instance, under-trust may be resulted from anchoring bias when humans look at only salient features of XAI class and thus judge the quality of the XAI class to be untrustworthy.…”

“…Inspiring trust can be done without necessarily improving users' mental models and beliefs of the true AI capability and limitations. On the other hand, calibrated trust may need more engagement from end-users to understand the AI system's reasoning (Naiseh et al, 2021c). We study the effect of four distinct XAI classes (Local, Example-based, Counterfactual, and Global explanations) on trust calibration during a collaborative Human-AI task.…”