Will You Accept the AI Recommendation? Predicting Human Behavior in AI-Assisted Decision Making

Wang, Xinru; Lu, Zhuoran; Yin, Ming

doi:10.1145/3485447.3512240

Cited by 17 publications

(4 citation statements)

References 96 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is because they tend to believe they are correct when they are confident (despite this is not always true as humans may sometimes suffer from the "Dunning-Kruger effect"). More specifically, it was found that when the performance information about the AI model is absent, humans are more likely to rely on their own judgement when their own decision confidence is high, while they are more receptive to AI recommendation when their own decision confidence is low (Wang, Lu, and Yin 2022;Chong et al 2022). In addition, the level of agreement between the AI recommendation and humans' independent judgement on those decision making tasks where humans are highly confident about their own judgement also significantly impacts humans' trust and reliance on AI-the higher the level of agreement, the more humans trust and rely on the AI recommendation (Lu and Yin 2021).…”

Section: Accounting For Engagement Behavior When Ai Assists Individua...mentioning

confidence: 99%

Accounting for Human Engagement Behavior to Enhance AI-Assisted Decision Making

Yin

2024

AAAI-SS

View full text Add to dashboard Cite

Artificial intelligence (AI) technologies have been increasingly integrated into human workflows. For example, the usage of AI-based decision aids in human decision-making processes has resulted in a new paradigm of AI-assisted decision making---that is, the AI-based decision aid provides a decision recommendation to the human decision makers, while humans make the final decision. The increasing prevalence of human-AI collaborative decision making highlights the need to understand how humans engage with the AI-based decision aid in these decision-making processes, and how to promote the effectiveness of the human-AI team in decision making. In this talk, I'll discuss a few examples illustrating that when AI is used to assist humans---both an individual decision maker or a group of decision makers---in decision making, people's engagement with the AI assistance is largely subject to their heuristics and biases, rather than careful deliberation of the respective strengths and limitations of AI and themselves. I'll then describe how to enhance AI-assisted decision making by accounting for human engagement behavior in the designs of AI-based decision aids. For example, AI recommendations can be presented to decision makers in a way that promotes their appropriate trust and reliance on AI by leveraging or mitigating human biases, informed by the analysis of human competence in decision making. Alternatively, AI-assisted decision making can be improved by developing AI models that can anticipate and adapt to the engagement behavior of human decision makers.

show abstract

Section: Accounting For Engagement Behavior When Ai Assists Individua...mentioning

confidence: 99%

Accounting for Human Engagement Behavior to Enhance AI-Assisted Decision Making

Yin

2024

AAAI-SS

View full text Add to dashboard Cite

show abstract

“…One approach to building humans' mental models is through data-driven methods. For example, in a loan approval task, Wang et al [95] construct a general human prediction model via a neural network with crowdsourcing data.…”

Section: Mental Model In Human-ai Collaborationmentioning

confidence: 99%

Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making

Ma¹,

Lü²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

In AI-assisted decision-making, it is critical for human decision-makers to know when to trust AI and when to trust themselves.However, prior studies calibrated human trust only based on AI confidence indicating AI's correctness likelihood (CL) but ignored humans' CL, hindering optimal team decision-making. To mitigate this gap, we proposed to promote humans' appropriate trust based on the CL of both sides at a task-instance level. We first modeled humans' CL by approximating their decision-making models and computing their potential performance in similar instances. We demonstrated the feasibility and effectiveness of our model via two preliminary studies. Then, we proposed three CL exploitation strategies to calibrate users' trust explicitly/implicitly in the AI-assisted decision-making process. Results from a between-subjects experiment (N=293) showed that our CL exploitation strategies promoted more appropriate human trust in AI, compared with only using AI confidence. We further provided practical implications for more human-compatible AI-assisted decision-making.CCS Concepts: • Human-centered computing → Empirical studies in HCI.

show abstract

“…In our second goal, we use the cognitive modeling approach to understand how a human's reliance policy depends on a number of factors related to the human and the AI. Previous research has shown that a human's confidence in their own decision influences the tendency to rely on AI assistance (Lu and Yin, 2021;Pescetelli et al, 2021;Wang et al, 2022). In addition, reliance on the AI is also affected by the AI's confidence in its decision (Zhang et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

AI-Assisted Decision-making: a Cognitive Modeling Approach to Infer Latent Reliance Strategies

et al. 2022

View full text Add to dashboard Cite

AI assistance is readily available to humans in a variety of decision-making applications. In order to fully understand the efficacy of such joint decision-making, it is important to first understand the human’s reliance on AI. However, there is a disconnect between how joint decision-making is studied and how it is practiced in the real world. More often than not, researchers ask humans to provide independent decisions before they are shown AI assistance. This is done to make explicit the influence of AI assistance on the human’s decision. We develop a cognitive model that allows us to infer the latent reliance strategy of humans on AI assistance without asking the human to make an independent decision. We validate the model’s predictions through two behavioral experiments. The first experiment follows a concurrent paradigm where humans are shown AI assistance alongside the decision problem. The second experiment follows a sequential paradigm where humans provide an independent judgment on a decision problem before AI assistance is made available. The model’s predicted reliance strategies closely track the strategies employed by humans in the two experimental paradigms. Our model provides a principled way to infer reliance on AI-assistance and may be used to expand the scope of investigation on human-AI collaboration.

show abstract

Will You Accept the AI Recommendation? Predicting Human Behavior in AI-Assisted Decision Making

Cited by 17 publications

References 96 publications

Accounting for Human Engagement Behavior to Enhance AI-Assisted Decision Making

Accounting for Human Engagement Behavior to Enhance AI-Assisted Decision Making

Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making

AI-Assisted Decision-making: a Cognitive Modeling Approach to Infer Latent Reliance Strategies

Contact Info

Product

Resources

About