The Effect of Interpretable Artificial Intelligence on Repeated Managerial Decision-Making Under Uncertainty

Altintas, Onur; Gu, Bin

doi:10.2139/ssrn.4331145

Cited by 1 publication

(1 citation statement)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interpretability is theorized as allowing "the user to rapidly calibrate their trust in the system's outputs, spotting flaws in its reasoning or seeing when it is unsure" [53]. Studies have found mixed effects of output interpretability [54,55,56,57] and model transparency [58,59], although field experiments on physicians find that they benefit from explainable AI advice [60,61]. We use verbal explanations in the form of a GPT model's text reasoning to test whether interpretability makes a difference.…”

Section: Literature Reviewmentioning

confidence: 99%

Taking Advice from ChatGPT

Zhang¹

2023

Preprint

View full text Add to dashboard Cite

A growing literature studies how humans incorporate advice from algorithms. This study examines an algorithm with millions of daily users: ChatGPT. We conduct a lab experiment in which 118 student participants answer 2,828 multiple-choice questions across 25 academic subjects. We present participants with answers from a GPT model and allow them to update their initial responses. We find that the advisor’s identity (``AI chatbot'' versus a human ``expert''), presence of written justification, and advice correctness do not significant affect weight on advice. Instead, we show that participants weigh advice more heavily if they (1) are unfamiliar with the topic, (2) used ChatGPT in the past, or (3) received more accurate advice previously. These three effects—task difficulty, algorithm familiarity, and experience, respectively—appear to be stronger with an AI chatbot as the advisor. Moreover, we find that participants are able to place greater weight on correct advice only when written justifications are provided. In a parallel analysis, we find that the student participants are miscalibrated and significantly underestimate the accuracy of ChatGPT on 10/25 topics. Students under-weigh advice by over 50% and would have scored better if they trusted ChatGPT more.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Taking Advice from ChatGPT

Zhang¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

The Effect of Interpretable Artificial Intelligence on Repeated Managerial Decision-Making Under Uncertainty

Cited by 1 publication

References 29 publications

Taking Advice from ChatGPT

Taking Advice from ChatGPT

Contact Info

Product

Resources

About