Explainable artificial intelligence in emergency medicine: an overview

Abstract

Artificial intelligence (AI) and machine learning (ML) have potential to revolutionize emergency medical care by enhancing triage systems, improving diagnostic accuracy, refining prognostication, and optimizing various aspects of clinical care. However, as clinicians often lack AI expertise, they might perceive AI as a “black box,” leading to trust issues. To address this, “explainable AI,” which teaches AI functionalities to end-users, is important. This review presents the definitions, importance, and role of explainable AI, as well as potential challenges in emergency medicine. First, we introduce the terms explainability, interpretability, and transparency of AI models. These terms sound similar but have different roles in discussion of AI. Second, we indicate that explainable AI is required in clinical settings for reasons of justification, control, improvement, and discovery and provide examples. Third, we describe three major categories of explainability: pre-modeling explainability, interpretable models, and post-modeling explainability and present examples (especially for post-modeling explainability), such as visualization, simplification, text justification, and feature relevance. Last, we show the challenges of implementing AI and ML models in clinical settings and highlight the importance of collaboration between clinicians, developers, and researchers. This paper summarizes the concept of “explainable AI” for emergency medicine clinicians. This review may help clinicians understand explainable AI in emergency contexts.

Drinking from the Holy Grail—Does a Perfect Triage System Exist? And Where to Look for It?

“…Researchers from Turkey showed that there is a strong conviction among ED staff about the benefits of using AI support to assess a patient during triage; however, concerns about the related ethical aspects of such an intervention are also present among them [ 63 ]. Some of the misunderstandings about the idea of machine learning are addressed by the publications discussing the topic in more depth in the emergency medicine group [ 64 ].…”