A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI

Abstract

Recently, artificial intelligence and machine learning in general have demonstrated remarkable performances in many tasks, from image processing to natural language processing, especially with the advent of deep learning (DL). Along with research progress, they have encroached upon many different fields and disciplines. Some of them require high level of accountability and thus transparency, for example, the medical sector. Explanations for machine decisions and predictions are thus needed to justify their reliability. This requires greater interpretability, which often means we need to understand the mechanism underlying the algorithms. Unfortunately, the blackbox nature of the DL is still unresolved, and many machine decisions are still poorly understood. We provide a review on interpretabilities suggested by different research works and categorize them. The different categories show different dimensions in interpretability research, from approaches that provide "obviously" interpretable information to the studies of complex patterns. By applying the same categorization to interpretability in medical research, it is hoped that: 1) clinicians and practitioners can subsequently approach these methods with caution; 2) insight into interpretability will be born with more considerations for medical practices; and 3) initiatives to push forward data-based, mathematically grounded, and technically grounded medical education are encouraged.

Detecting and Reversing Myocardial Ischemia Using an Artificially Intelligent Bioelectronic Medicine

“…AI enabled medicines can suffer from a lack of interpretability – where either data or algorithm decisions cannot be readily understood. AI enabled medicines must be easily interpretable for widespread adoption (Vellido, 2019; Tonekaboni et al, 2019; Tjoa, E., & Guan, 2019). Visualizations are one solution for creating interpretable representations of both high-dimensional data and complex algorithm decisions.…”

“…Lastly, artificial intelligence (AI) is becoming a powerful tool in medicine. Importantly, AI enabled medicines must be easily interpretable for widespread adoption (Vellido, 2019; Tonekaboni et al, 2019; Tjoa, E., & Guan, 2019). AI interpretability can be enhanced using visualizations.…”

Interpretation of cluster structures in pain‐related phenotype data using explainable artificial intelligence (XAI)

“…On the other hand, an attempt is made to increase the complexity of XAI models (Arrieta et al., 2019). However, biomedical studies that compare different existing models and provide meaningful conclusions about their interpretability contribute to this research area (Tjoa & Guan, 2019). The present report contributes to this by proposing a comparative scenario for the selection of a suitable XAI, since no prior selection can be recommended, considering that the XAIs already in a limited number of data sets did not show a consistent order of classification performance.…”

Academics as leaders in the cancer artificial intelligence revolution

“…It is still not easy to explain exactly what a neural network learns during training for any particular classification task because of the complexity of the algorithm (the black box problem 29 ). All methods strive to reflect the patterns the model learned from the data; examples include synthetic images generated by general adversarial networks, t‐distributed stochastic neighbor embedding (t‐SNE) clustering of feature layers to find meaningful groups, and the identification of concepts of interest that can be labeled as a specific feature 30 . A caveat of the black‐box explainability approach is that it is using 1 model to explain a second, and the models can be unreliable and misleading.…”

“…One solution is to use of the aforementioned explainability tools and/or interpretable model approaches to help identify which patterns the model is learning 29‐31 . Another solution, recommended by Riley, is to ask the model to predict other things.…”