Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making

Abstract

In AI-assisted decision-making, it is critical for human decision-makers to know when to trust AI and when to trust themselves.However, prior studies calibrated human trust only based on AI confidence indicating AI's correctness likelihood (CL) but ignored humans' CL, hindering optimal team decision-making. To mitigate this gap, we proposed to promote humans' appropriate trust based on the CL of both sides at a task-instance level. We first modeled humans' CL by approximating their decision-making models and computing their potential performance in similar instances. We demonstrated the feasibility and effectiveness of our model via two preliminary studies. Then, we proposed three CL exploitation strategies to calibrate users' trust explicitly/implicitly in the AI-assisted decision-making process. Results from a between-subjects experiment (N=293) showed that our CL exploitation strategies promoted more appropriate human trust in AI, compared with only using AI confidence. We further provided practical implications for more human-compatible AI-assisted decision-making.CCS Concepts: • Human-centered computing → Empirical studies in HCI.

A Diachronic Perspective on User Trust in AI under Uncertainty

“…Optimizing for cooperation with humans is more productive than focusing solely on model performance (Bansal et al, 2021a). Human-AI collaboration research has focused on AI systems explaining their predictions (Ribeiro et al, 2016) or examining the relationship between trust and AI system's accuracy (Rechkemmer and Yin, 2022;Ma et al, 2023). Related to our work, Papenmeier et al (2019); Bansal et al (2021b); Wang and Yin (2022); Papenmeier et al (2022) examined the influence of explanations and found that inaccurate ones act as deceptive experiences which erode trust.…”