On Assessing Trustworthy AI in Healthcare. Machine Learning as a Supportive Tool to Recognize Cardiac Arrest in Emergency Calls

Abstract

Artificial Intelligence (AI) has the potential to greatly improve the delivery of healthcare and other services that advance population health and wellbeing. However, the use of AI in healthcare also brings potential risks that may cause unintended harm. To guide future developments in AI, the High-Level Expert Group on AI set up by the European Commission (EC), recently published ethics guidelines for what it terms “trustworthy” AI. These guidelines are aimed at a variety of stakeholders, especially guiding practitioners toward more ethical and more robust applications of AI. In line with efforts of the EC, AI ethics scholarship focuses increasingly on converting abstract principles into actionable recommendations. However, the interpretation, relevance, and implementation of trustworthy AI depend on the domain and the context in which the AI system is used. The main contribution of this paper is to demonstrate how to use the general AI HLEG trustworthy AI guidelines in practice in the healthcare domain. To this end, we present a best practice of assessing the use of machine learning as a supportive tool to recognize cardiac arrest in emergency calls. The AI system under assessment is currently in use in the city of Copenhagen in Denmark. The assessment is accomplished by an independent team composed of philosophers, policy makers, social scientists, technical, legal, and medical experts. By leveraging an interdisciplinary team, we aim to expose the complex trade-offs and the necessity for such thorough human review when tackling socio-technical applications of AI in healthcare. For the assessment, we use a process to assess trustworthy AI, called 1Z-Inspection® to identify specific challenges and potential ethical trade-offs when we consider AI in practice.

Co-Design of a Trustworthy AI System in Healthcare: Deep Learning Based Skin Lesion Classifier

“…The findings show that several socio-environmental factors, which impact model performance, could not be foreseen during development. Our previous work within Z-inspection ® (Zicari, et al, 2021b), makes a similar finding when assessing a case where AI is used to detect sound patterns of callers to an emergency line, to indicate the probability of a cardiac arrest situation to an operator. The descriptive statistics of the modeling solution show improved ability for detection; however, a further examination of the collaborative system shows that operator trust toward the system is low (Blomberg et al, 2021).…”

“…Our approach uses a holistic process, called Z-inspection ® (Zicari, et al, 2021b), to help assisting engineers in the early co-design of an AI system to satisfy the requirement for Trustworthy AI as defined by the High-Level Expert Group on AI (AI HLEG) set up by the European Commission. One of the key features of the Z-inspection ® is the involvement of a multidisciplinary team of experts co-creating together with the AI engineers, their managers to ensure that the AI system is trustworthy.…”

From the ground truth up: doing AI ethics from practice to principles

“…Going from practice to principles starts with cases. The two discussed here were produced by a team of philosophers, computer scientists, lawyers, and doctors organized out of the Frankfurt Big Data Lab in Germany (Zicari et al 2021a, Zicari et al 2021b. Working with algorithmic startup companies, we collaboratively explore their development experiences, with attention split between ethics, technology, law, and medicine (Brusseau 2020).…”

“…If blue lips are mentioned, the probability also rises. If both, the alarm illuminates (Zicari et al 2021a). However, beyond that and a few similar anecdotes, there was no human-oriented discussion of the AI process, nothing that would make sense to a doctor.…”

“…When the technology was implemented, it produced unsurprising and also surprising results. Unsurprisingly, true cardiac arrest was recognized more frequently and quickly by the artificial intelligence than by its human partner (Zicari et al 2021a). However, the machine was also less specific: the AI returned many false positive alerts which were largely ignored by the dispatchers.…”