Color for Characters - Effects of Visual Explanations of AI on Trust and Observability

Schrills, Tim; Franke, Thomas

doi:10.1007/978-3-030-50334-5_8

Cited by 4 publications

(18 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, the next step is for practitioners to adopt them into use. While the research findings seem promising concerning being able to, for example, increase users' trust in AI systems (Ehsan et al, 2019;Schrills and Franke, 2020;Wang et al, 2019;Weitz et al, 2019b;Xie et al, 2019;Yin et al, 2019) and feeling of control (Ngo et al, 2020;Oh et al, 2018 of the AI system, these results can be different under real-world circumstances. Thus, practitioners are encouraged to take promising design recommendations and adapt them into practice, but measure their effects on end users.…”

Section: Practical Implicationsmentioning

confidence: 97%

“…Yet, the term trustworthiness appeared ubiquitously in the selected studies referring to a characteristic of the AI system. However, the empirical studies focused on end users' trust in the system, which of course was influenced by how end users received knowledge about the systemmeaning communication (Brennen, 2020;Bussone et al, 2015;Cheng et al, 2019;Ehsan et al, 2019;Schrills and Franke, 2020;Yin et al, 2019). Table 4.…”

Section: End Users and Application Contexts Of Ai Systemsmentioning

confidence: 99%

“…As pointed out earlier, transparency was closely related to the discussion on trust, as the lack of transparency can adversely impact trust (Ehsan et al, 2019) but also understandability, as perceived transparency is connected to how well users can understand content (Cramer et al, 2008). While transparency could be measured objectively as a characteristic of the system, in the reviewed literature, transparency was primarily scrutinized from the viewpoint of end users (Brennen, 2020;Schrills and Franke, 2020;Ngo et al, 2020;Eiband et al, 2018). While some studies were conducted with AI systems created specifically for research purposes (e.g.…”

Section: Xai Objectivesmentioning

confidence: 99%

“…Several studies (Chazette and Schneider, 2020;Cramer et al, 2008;Oh et al, 2018) argued that always displaying explanations to end users would reduce the usability and even understandability of AI systems and that it would be counterproductive to force explanations to all AI systems. Thus, AI system designers should consider the UI design as a key component for understandability and test for most effective ways to display and visualize explanations (Eslami et al, 2018;Ngo et al, 2020;Schrills and Franke, 2020;Weitz et al, 2019a, b).…”

Section: Design Recommendations For Explaining Ai Systems To End Usersmentioning

confidence: 99%

See 3 more Smart Citations

How to explain AI systems to end users: a systematic literature review and research agenda

Laato

Tiainen

Islam³

et al. 2022

INTR

View full text Add to dashboard Cite

PurposeInscrutable machine learning (ML) models are part of increasingly many information systems. Understanding how these models behave, and what their output is based on, is a challenge for developers let alone non-technical end users.Design/methodology/approachThe authors investigate how AI systems and their decisions ought to be explained for end users through a systematic literature review.FindingsThe authors’ synthesis of the literature suggests that AI system communication for end users has five high-level goals: (1) understandability, (2) trustworthiness, (3) transparency, (4) controllability and (5) fairness. The authors identified several design recommendations, such as offering personalized and on-demand explanations and focusing on the explainability of key functionalities instead of aiming to explain the whole system. There exists multiple trade-offs in AI system explanations, and there is no single best solution that fits all cases.Research limitations/implicationsBased on the synthesis, the authors provide a design framework for explaining AI systems to end users. The study contributes to the work on AI governance by suggesting guidelines on how to make AI systems more understandable, fair, trustworthy, controllable and transparent.Originality/valueThis literature review brings together the literature on AI system communication and explainable AI (XAI) for end users. Building on previous academic literature on the topic, it provides synthesized insights, design recommendations and future research agenda.

show abstract

Section: Practical Implicationsmentioning

confidence: 97%

Section: End Users and Application Contexts Of Ai Systemsmentioning

confidence: 99%

Section: Xai Objectivesmentioning

confidence: 99%

Section: Design Recommendations For Explaining Ai Systems To End Usersmentioning

confidence: 99%

See 2 more Smart Citations

How to explain AI systems to end users: a systematic literature review and research agenda

Laato

Tiainen

Islam³

et al. 2022

INTR

View full text Add to dashboard Cite

show abstract

“…Habits reduce sensitivity to minor changes in the explanation interface, curtail explanation utilisation, and reduce assessment and reflection about the decision [23]. For example, pharmacists who are in the habit of making decisions using AI-based screening prescription tool may fail to recognise incomplete explanation [72]. Previous experiment [29] exposed participants repeatedly to pictures of people to form well-practised reactions toward them.…”

Section: Design For Challenging Habitual Actionsmentioning

confidence: 99%

Explainable recommendation: when design meets trust calibration

et al. 2021

View full text Add to dashboard Cite

Human-AI collaborative decision-making tools are being increasingly applied in critical domains such as healthcare. However, these tools are often seen as closed and intransparent for human decision-makers. An essential requirement for their success is the ability to provide explanations about themselves that are understandable and meaningful to the users. While explanations generally have positive connotations, studies showed that the assumption behind users interacting and engaging with these explanations could introduce trust calibration errors such as facilitating irrational or less thoughtful agreement or disagreement with the AI recommendation. In this paper, we explore how to help trust calibration through explanation interaction design. Our research method included two main phases. We first conducted a think-aloud study with 16 participants aiming to reveal main trust calibration errors concerning explainability in AI-Human collaborative decision-making tools. Then, we conducted two co-design sessions with eight participants to identify design principles and techniques for explanations that help trust calibration. As a conclusion of our research, we provide five design principles: Design for engagement, challenging habitual actions, attention guidance, friction and support training and learning. Our findings are meant to pave the way towards a more integrated framework for designing explanations with trust calibration as a primary goal.

show abstract

Explanation matters: An experimental study on explainable AI

Hamm,

Klesel,

Coberger

et al. 2023

Electron Markets

View full text Add to dashboard Cite

Explainable artificial intelligence (XAI) is an important advance in the field of machine learning to shed light on black box algorithms and thus a promising approach to improving artificial intelligence (AI) adoption. While previous literature has already addressed the technological benefits of XAI, there has been little research on XAI from the user’s perspective. Building upon the theory of trust, we propose a model that hypothesizes that post hoc explainability (using Shapley Additive Explanations) has a significant impact on use-related variables in this context. To test our model, we designed an experiment using a randomized controlled trial design where participants compare signatures and detect forged signatures. Surprisingly, our study shows that XAI only has a small but significant impact on perceived explainability. Nevertheless, we demonstrate that a high level of perceived explainability has a strong impact on important constructs including trust and perceived usefulness. A post hoc analysis shows that hedonic factors are significantly related to perceived explainability and require more attention in future research. We conclude with important directions for academia and for organizations.

show abstract

Color for Characters - Effects of Visual Explanations of AI on Trust and Observability

Cited by 4 publications

References 39 publications

How to explain AI systems to end users: a systematic literature review and research agenda

How to explain AI systems to end users: a systematic literature review and research agenda

Explainable recommendation: when design meets trust calibration

Explanation matters: An experimental study on explainable AI

Contact Info

Product

Resources

About