Trust in Clinical AI: Expanding the Unit of Analysis

Abstract

From diagnosis to patient scheduling, AI is increasingly being considered across different clinical applications. Despite increasingly powerful clinical AI, uptake into actual clinical workflows remains limited. One of the major challenges is developing appropriate trust with clinicians. In this paper, we investigate trust in clinical AI in a wider perspective beyond user interactions with the AI. We offer several points in the clinical AI development, usage, and monitoring process that can have a significant impact on trust. We argue that the calibration of trust in AI should go beyond explainable AI and focus on the entire process of clinical AI deployment. We illustrate our argument with case studies from practitioners implementing clinical AI in practice to show how trust can be affected by different stages in the deployment cycle.

Tweaking algorithms. Technopolitical issues associated with artificial intelligence based tuberculosis detection in global health

“…As analyzed in the literature, the level of trust placed in AI systems conditions the use that can be made of them. 10 What is known as calibration is also at play in the expectations raised by usage compared with the actual possibilities of the tools using AI. Two biases are often considered in practice: A so-called over-reliance bias and an algorithmic aversion bias, i.e., a complete refusal to use algorithms.…”