Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

Abstract

Background. Artificial intelligence (AI) has developed rapidly, and its application extends to clinical decision support system (CDSS) for improving healthcare quality. However, the interpretability of AI-driven CDSS poses significant challenges to widespread application. Objective. This study is a review of the knowledge-based and data-based CDSS literature regarding interpretability in health care. It highlights the relevance of interpretability for CDSS and the area for improvement from technological and medical perspectives. Methods. A systematic search was conducted on the interpretability-related literature published from 2011 to 2020 and indexed in the five databases: Web of Science, PubMed, ScienceDirect, Cochrane, and Scopus. Journal articles that focus on the interpretability of CDSS were included for analysis. Experienced researchers also participated in manually reviewing the selected articles for inclusion/exclusion and categorization. Results. Based on the inclusion and exclusion criteria, 20 articles from 16 journals were finally selected for this review. Interpretability, which means a transparent structure of the model, a clear relationship between input and output, and explainability of artificial intelligence algorithms, is essential for CDSS application in the healthcare setting. Methods for improving the interpretability of CDSS include ante-hoc methods such as fuzzy logic, decision rules, logistic regression, decision trees for knowledge-based AI, and white box models, post hoc methods such as feature importance, sensitivity analysis, visualization, and activation maximization for black box models. A number of factors, such as data type, biomarkers, human-AI interaction, needs of clinicians, and patients, can affect the interpretability of CDSS. Conclusions. The review explores the meaning of the interpretability of CDSS and summarizes the current methods for improving interpretability from technological and medical perspectives. The results contribute to the understanding of the interpretability of CDSS based on AI in health care. Future studies should focus on establishing formalism for defining interpretability, identifying the properties of interpretability, and developing an appropriate and objective metric for interpretability; in addition, the user's demand for interpretability and how to express and provide explanations are also the directions for future research.

Continuum Robots and Magnetic Soft Robots: From Models to Interdisciplinary Challenges for Medical Applications

“…An interdisciplinary perspective, particularly from fields such as computer science, ethics in artificial intelligence, and psychology, offers new directions and methodologies for addressing the issue of model interpretability [ 151 , 152 , 153 ]. Integrating concepts like attention mechanisms [ 154 ] and local interpretable models can uncover the rationale behind model decisions [ 155 ].…”

Revolutionizing Healthcare with Smarter AI: In-depth Exploration of Advancements, Challenges, and Future Directions

“…The integration of Artificial Intelligence (AI) into medical healthcare systems has acquired important attentiveness from policymakers, researchers, and practitioners correspondingly. This fragment reviews current literature to postulate a comprehensive understanding of the existing state of expertise observing the ethical associations and scenarios of AI in healthcare [17,18]. The researchers highlight how AI is revolutionizing several aspects of healthcare.…”

A Systematic Literature Review on Transparencyand Interpretability of AI models in Healthcare:Taxonomies, Tools, Techniques, Datasets, OpenResearch Challenges, and Future Trends

“…It evaluated interpretability techniques such as SHAP, Grad-CAM, and LIME and discussed their usability and reliability, aiming to advance XAI in healthcare for researchers and professionals. Also, Authors in [17] reviewed AI-driven CDSS from 2011 to 2020, highlighting the value of interpretability, exploring techniques, and stressing its research prospect in healthcare applications. Similarly, the research study [42] attended demonstrates the significance of openness in AI-driven healthcare tools, providing a framework to quantify transparency and reliability.…”

“…A transparent AI design in healthcare offers vibrant explanations for its diagnostic recommendations, detailing the functions or patterns in client data that led to a specific forecast [15], [16]. Interpretability, on the other hand, distillates human users (doctors) capability to interpret and understand the outputs produced by an AI design [17]. An interpretable AI design supplies insights into how it arrives at its conclusions in an intuitive and significant way to users [18], [19].…”