The Next Generation of Medical Decision Support: A Roadmap Toward Transparent Expert Companions

Abstract

Increasing quality and performance of artificial intelligence (AI) in general and machine learning (ML) in particular is followed by a wider use of these approaches in everyday life. As part of this development, ML classifiers have also gained more importance for diagnosing diseases within biomedical engineering and medical sciences. However, many of those ubiquitous high-performing ML algorithms reveal a black-box-nature, leading to opaque and incomprehensible systems that complicate human interpretations of single predictions or the whole prediction process. This puts up a serious challenge on human decision makers to develop trust, which is much needed in life-changing decision tasks. This paper is designed to answer the question how expert companion systems for decision support can be designed to be interpretable and therefore transparent and comprehensible for humans. On the other hand, an approach for interactive ML as well as human-in-the-loop-learning is demonstrated in order to integrate human expert knowledge into ML models so that humans and machines act as companions within a critical decision task. We especially address the problem of Semantic Alignment between ML classifiers and its human users as a prerequisite for semantically relevant and useful explanations as well as interactions. Our roadmap paper presents and discusses an interdisciplinary yet integrated Comprehensible Artificial Intelligence (cAI)-transition-framework with regard to the task of medical diagnosis. We explain and integrate relevant concepts and research areas to provide the reader with a hands-on-cookbook for achieving the transition from opaque black-box models to interactive, transparent, comprehensible and trustworthy systems. To make our approach tangible, we present suitable state of the art methods with regard to the medical domain and include a realization concept of our framework. The emphasis is on the concept of Mutual Explanations (ME) that we introduce as a dialog-based, incremental process in order to provide human ML users with trust, but also with stronger participation within the learning process.

Effects of a Differential Diagnosis List of Artificial Intelligence on Differential Diagnoses by Physicians: An Exploratory Analysis of Data from a Randomized Controlled Study

“…Indeed, some participants noted, "Some of my diagnoses might have been affected by the AI list when I could not judge whether AI was correct or not." To enhance the physicians' skill in discriminating whether the AI is correct or not, one possible solution may be to visualize the process and evidence in the generation of AI differential diagnoses (opening the black box of AI), which is called explainable AI [24,25]. Explainable AI refers to, in a nutshell, AI in which the output by the AI can be logically understood and explained by humans.…”

“…Some participants in the study said, “In some cases, I felt that the list by AI was wrong this was from my intuition, though.” However, as many as 56% of physician diagnoses were identical with the AI diagnoses in the AI-incorrect cases, the physicians’ discrimination skill was too far from ideal. Indeed, some participants noted, “Some of my diagnoses might have been affected by the AI list when I could not judge whether AI was correct or not.” To enhance the physicians’ skill in discriminating whether the AI is correct or not, one possible solution may be to visualize the process and evidence in the generation of AI differential diagnoses (opening the black box of AI), which is called explainable AI [ 24 , 25 ]. Explainable AI refers to, in a nutshell, AI in which the output by the AI can be logically understood and explained by humans.…”

Explanation as a process: user-centric construction of multi-level and multi-modal explanations

“…Since our approach is interpretable, it could help users in the future to uncover causal-ities between data and a system's prediction. This is especially important in decision-critical areas, such as medicine [14,4,31].…”

Explainable Artificial Intelligence for Tabular Data: A Survey

“…Bruckert et al [212] argued that semantic and contextual information must be taken into account while generating explanations. They also argued that human interpretable explanations must shed light on logical as well as causal correlations.…”