The robot butler: How and why should we study predictive algorithms and artificial intelligence (AI) in healthcare?

Abstract

Artificial intelligence (AI) and algorithms are heralded as significant solutions to the widening gap between the rising healthcare needs of ageing and multi-morbid populations and the scarcity of resources to provide such care. Objective This article investigates how the PMHnet algorithm – an AI prognostication tool developed in Denmark to predict the one-year all-cause mortality risk for patients hospitalized with ischemic heart disease – was presented to cardiologists working in the hospital setting, and how they responded to this novel decision-support tool. Methods Empirically, we draw upon ethnographic fieldwork in the Danish-led international research project, PM Heart, which since 2019 has developed the PMHnet algorithm and implemented the software into the electronic health record system in hospitals in Eastern Denmark (the Capital Region and Region Zealand). Results Paying careful attention to the hopes and concerns of cardiologists who will have to embrace and adapt to algorithmic tools in their everyday work of diagnosing and treating patients, we identify three analytical themes meriting attention when AI is implemented in healthcare: 1) the re-negotiation of agency and autonomy in human-algorithm relations, 2) accountability in algorithmic prognostication and 3) the complex relationship between association and causation actualized by predictive algorithms. From these analytical themes, we elicit methodological questions to guide future ethnographic explorations of how AI and advanced algorithms are put to use in the healthcare system, with what implications, and for whom. Conclusion We conclude that local, qualitative investigations of how algorithms are used, embraced and contested in everyday clinical practice are needed in order to understand their implications – good and bad, intended and unintended – for clinicians, patients and healthcare provision.