How do visual explanations foster end users' appropriate trust in machine learning?

Yang, Fumeng; Huang, Zhenfeng; Scholtz, Jean; Arendt, Dustin

doi:10.1145/3377325.3377480

Cited by 120 publications

(69 citation statements)

References 101 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An essential step towards this goal is to use explanations to guide people to trust an AI model when it is right and not to trust it when it is wrong. In other words, with the assistance of model explanations, people should have better capability of calibrating their trust in the model [69,71]. Note that when an explanation simply improves the human-AI joint decision making accuracy, it does not necessarily mean this desideratum is satisfied.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Lai et al [41] deception detection feature contribution N/A N/A ✓? Cai et al [13] drawing recognition example-based mixed results N/A N/A Yang et al [69] leaf classification example-based N/A N/A ✓ Note: "N/A" means the study does not examine the desideratum. ✓ (or ✗) means the study finds (or does not find) evidence suggesting the explanation method it examines satisfies a desideratum.…”

Section: Literature Reviewmentioning

confidence: 99%

“…To answer these questions, researchers have been advocating for moving beyond defining what constitutes a "good" explanation using model designer's intuition but actually examining how useful an explanation is with human users [24,59]. In responding to this call, there is recently a growing line of literature on empirically evaluating the effectiveness of XAI methods (e.g., [14,16,41,69,71]). Yet, principles required for an explanation to be considered helpful in AI-assisted decision making, arguably, still remain to be articulated and comprehensively assessed.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

Wang

Yin

2021

26th International Conference on Intelligent User Interfaces

155

View full text Add to dashboard Cite

This paper contributes to the growing literature in empirical evaluation of explainable AI (XAI) methods by presenting a comparison on the effects of a set of established XAI methods in AI-assisted decision making. Specifically, based on our review of previous literature, we highlight three desirable properties that ideal AI explanations should satisfy-improve people's understanding of the AI model, help people recognize the model uncertainty, and support people's calibrated trust in the model. Through randomized controlled experiments, we evaluate whether four types of common model-agnostic explainable AI methods satisfy these properties on two types of decision making tasks where people perceive themselves as having different levels of domain expertise in (i.e., recidivism prediction and forest cover prediction). Our results show that the effects of AI explanations are largely different on decision making tasks where people have varying levels of domain expertise in, and many AI explanations do not satisfy any of the desirable properties for tasks that people have little domain expertise in. Further, for decision making tasks that people are more knowledgeable, feature contribution explanation is shown to satisfy more desiderata of AI explanations, while the explanation that is considered to resemble how human explain decisions (i.e., counterfactual explanation) does not seem to improve calibrated trust. We conclude by discussing the implications of our study for improving the design of XAI methods to better support human decision making. CCS CONCEPTS• Human-centered computing → Empirical studies in HCI; • Computing methodologies → Machine learning.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

Wang

Yin

2021

26th International Conference on Intelligent User Interfaces

155

View full text Add to dashboard Cite

show abstract

“…Many applications involving machine and deep learning algorithms provide post-hoc explanations of why a decision refused mortgage or parole requests. However, exploratory user interfaces using interactive visual designs offer a more likely path to successful customer adoption and acceptance (Chatzimparmpas et al, 2020;Hohman et al, 2018;Nourashrafeddin et al, 2018;Yang et al, 2020). Well-designed interactive visual interfaces will improve the work of machine learning algorithm developers and facilitate comprehension by various stakeholders.…”

Section: Figure 7 Cliché-ridden Images Of Humanoid Robot Hands and Smentioning

confidence: 99%

Human-Centered Artificial Intelligence: Three Fresh Ideas

Shneiderman¹

2020

THCI

188

View full text Add to dashboard Cite

Human-Centered AI (HCAI) is a promising direction for designing AI systems that support human self-efficacy, promote creativity, clarify responsibility, and facilitate social participation. These human aspirations also encourage consideration of privacy, security, environmental protection, social justice, and human rights. This commentary reverses the current emphasis on algorithms and AI methods, by putting humans at the center of systems design thinking, in effect, a second Copernican Revolution. It offers three ideas: (1) a two-dimensional HCAI framework, which shows how it is possible to have both high levels of human control AND high levels of automation, (2) a shift from emulating humans to empowering people with a plea to shift language, imagery, and metaphors away from portrayals of intelligent autonomous teammates towards descriptions of powerful tool-like appliances and tele-operated devices, and (3) a three-level governance structure that describes how software engineering teams can develop more reliable systems, how managers can emphasize a safety culture across an organization, and how industry-wide certification can promote trustworthy HCAI systems. These ideas will be challenged by some, refined by others, extended to accommodate new technologies, and validated with quantitative and qualitative research. They offer a reframe-a chance to restart design discussions for products and services-which could bring greater benefits to individuals, families, communities, businesses, and society.

show abstract

“…The generated natural language rationales outperformed a baseline on ratings of Confidence, Human-Like, Adequately Justified, and Understandable with human judges when observing AI play traces of the arcade game alongside different text rationales. In general, many XAI studies test for user trust in the model to make correct decisions [18,40] and user confidence in the decision-making process [1,17]. Conversely, our study measures intelligibility to an outside observer of a complex, multi-agent system where players act and react to each other while pursuing adversarial goals in real time.…”

Section: Background and Related Workmentioning

confidence: 99%

Wait, But Why?: Assessing Behavior Explanation Strategies for Real-Time Strategy Games

Robertson

Kokkinakis

Hook

et al. 2021

26th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

Work in AI-based explanation systems has uncovered an interesting contradiction: people prefer and learn best from why explanations but expert esports commentators primarily answer what questions when explaining complex behavior in real-time strategy games. Three possible explanations for this contradiction are: 1.) broadcast audiences are well-informed and do not need why explanations; 2.) consuming why explanations in real-time is too cognitively demanding for audiences; or 3.) producing live why explanations is too difficult for commentators. We answer this open question by investigating the effects of explanation types and presentation modalities on audience recall and cognitive load in the context of an esports broadcast. We recruit 111 Dota 2 players and split them into three groups: the first group views a Dota 2 broadcast, the second group has the addition of an interactive map that provides what explanations, and the final group receives the interactive map with detailed why explanations. We find that participants who receive short interactive text prompts that provide what explanations outperform the no explanation group on a multiple-choice recall task. We also find that participants who receive detailed why explanations submit reports of cognitive load that are higher than the no explanation group. Our evidence supports the conclusion

show abstract

How do visual explanations foster end users' appropriate trust in machine learning?

Cited by 120 publications

References 101 publications

Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

Human-Centered Artificial Intelligence: Three Fresh Ideas

Wait, But Why?: Assessing Behavior Explanation Strategies for Real-Time Strategy Games

Contact Info

Product

Resources

About