Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations

Sperrle, Fabian; El-Assady, Mennatallah; Guo, Grace; Chau, Duen Horng; Endert, Alex; Keim, Daniel

doi:10.48550/arxiv.2009.06433

Cited by 5 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, a XAI component that helps users to better understand what the AI is doing has the potential to increase users' perceived trust. This is consistent with previous research that has shown that trust can only be built if humans understand the decisions of AI algorithms (Sperrle et al, 2020). Consequently, explanations help to verify predictions, improve models, and gain new insights into the problem which ultimately leads to an increase in trust in AI algorithms (Adadi & Berrada, 2018).…”

Section: Hypothesis Developmentsupporting

confidence: 90%

“…The fundamental idea is that individuals perceiving a higher degree of transparency by means of an explainable component will also perceive the outcome more positively. For this reason, the theory of trust (Gefen et al, 2003) has been used as a central theoretical lens to study the impact of XAI (e.g., Ribeiro et al, 2016;Sperrle et al, 2020;Thiebes et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Explanation matters: An experimental study on explainable AI

Hamm,

Klesel,

Coberger

et al. 2023

Electron Markets

View full text Add to dashboard Cite

Explainable artificial intelligence (XAI) is an important advance in the field of machine learning to shed light on black box algorithms and thus a promising approach to improving artificial intelligence (AI) adoption. While previous literature has already addressed the technological benefits of XAI, there has been little research on XAI from the user’s perspective. Building upon the theory of trust, we propose a model that hypothesizes that post hoc explainability (using Shapley Additive Explanations) has a significant impact on use-related variables in this context. To test our model, we designed an experiment using a randomized controlled trial design where participants compare signatures and detect forged signatures. Surprisingly, our study shows that XAI only has a small but significant impact on perceived explainability. Nevertheless, we demonstrate that a high level of perceived explainability has a strong impact on important constructs including trust and perceived usefulness. A post hoc analysis shows that hedonic factors are significantly related to perceived explainability and require more attention in future research. We conclude with important directions for academia and for organizations.

show abstract

Section: Hypothesis Developmentsupporting

confidence: 90%

Section: Related Workmentioning

confidence: 99%

Explanation matters: An experimental study on explainable AI

Hamm,

Klesel,

Coberger

et al. 2023

Electron Markets

View full text Add to dashboard Cite

show abstract

“…We focus on studies with the goal of evaluating, understanding and/or improving human performance and experience for a decision making task, rather than improving the model. This scope differentiates from prior surveys on empirical studies of human-AI interactions that either deviate from the scope of decision making or focus on only one aspect of it, like trust [63,89,126,127,141,146]. With the above-mentioned challenges in mind, our survey focuses on analyzing three aspects of study design choices made in these surveyed papers: the decision tasks, the types of AI and AI assistance elements, and the evaluation metrics.…”

Section: Introductionmentioning

confidence: 99%

Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies

Lai¹,

Chen²,

Liao³

et al. 2021

Preprint

View full text Add to dashboard Cite

As AI systems demonstrate increasingly strong predictive performance, their adoption has grown in numerous domains. However, in high-stakes domains such as criminal justice and healthcare, full automation is often not desirable due to safety, ethical, and legal concerns, yet fully manual approaches can be inaccurate and time consuming. As a result, there is growing interest in the research community to augment human decision making with AI assistance. Besides developing AI technologies for this purpose, the emerging field of human-AI decision making must embrace empirical approaches to form a foundational understanding of how humans interact and work with AI to make decisions. To invite and help structure research efforts towards a science of understanding and improving human-AI decision making, we survey recent literature of empirical human-subject studies on this topic. We summarize the study design choices made in over 100 papers in three important aspects: (1) decision tasks, (2) AI models and AI assistance elements, and(3) evaluation metrics. For each aspect, we summarize current trends, discuss gaps in current practices of the field, and make a list of recommendations for future research. Our survey highlights the need to develop common frameworks to account for the design and research spaces of human-AI decision making, so that researchers can make rigorous choices in study design, and the research community can build on each other's work and produce generalizable scientific knowledge. We also hope this survey will serve as a bridge for HCI and AI communities to work together to mutually shape the empirical science and computational technologies for human-AI decision making.

show abstract

“…In future work, we aim to systematically evaluate our approach with empirical user studies, recognizing design dimensions of XAI put forth by Sperrle et al [33]. In this context, our multi-modal and multi-level approach allows for precise control and manipulation of experimental conditions.…”

Section: Discussionmentioning

confidence: 99%

Explanation as a process: user-centric construction of multi-level and multi-modal explanations

Finzel,

Tafler,

Scheele

et al. 2021

Preprint

View full text Add to dashboard Cite

In the last years, XAI research has mainly been concerned with developing new technical approaches to explain deep learning models. Just recent research has started to acknowledge the need to tailor explanations to different contexts and requirements of stakeholders. Explanations must not only suit developers of models, but also domain experts as well as end users. Thus, in order to satisfy different stakeholders, explanation methods need to be combined. While multi-modal explanations have been used to make model predictions more transparent, less research has focused on treating explanation as a process, where users can ask for information according to the level of understanding gained at a certain point in time. Consequently, an opportunity to explore explanations on different levels of abstraction should be provided besides multi-modal explanations. We present a process-based approach that combines multi-level and multi-modal explanations. The user can ask for textual explanations or visualizations through conversational interaction in a drill-down manner. We use Inductive Logic Programming, an interpretable machine learning approach, to learn a comprehensible model. Further, we present an algorithm that creates an explanatory tree for each example for which a classifier decision is to be explained. The explanatory tree can be navigated by the user to get answers of different levels of detail. We provide a proof-of-concept implementation for concepts induced from a semantic net about living beings.

show abstract

Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations

Cited by 5 publications

References 26 publications

Explanation matters: An experimental study on explainable AI

Explanation matters: An experimental study on explainable AI

Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies

Explanation as a process: user-centric construction of multi-level and multi-modal explanations

Contact Info

Product

Resources

About