How to Induce Trust in Medical AI Systems

Reimer, Ulrich; Tödtli, Beat; Maier, Edith

doi:10.1007/978-3-030-65847-2_1

Cited by 4 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The practical ethical problem is the heterogeneity and lack of "gold standards" of comparable training and testing datasets for any particular problem with statistical and heuristic approaches, which do not take into account the wide range of qualitative differences between different types of data and knowledge, and the implementation of responsible evaluatory judgments by experts which cannot be reduced to simple measures. Complementary ap-proaches for this might be official certifications or criteria for controlling data biases, beside explainability, in order to build trust on these systems [128,129].…”

Section: Controversy Around Current Xai Methodsmentioning

confidence: 99%

A Scoping Review on the Progress, Applicability, and Future of Explainable Artificial Intelligence in Medicine

González-Alday,

García-Cuesta,

Kulikowski

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Due to the success of artificial intelligence (AI) applications in the medical field over the past decade, concerns about the explainability of these systems have increased. The reliability requirements of black-box algorithms for making decisions affecting patients pose a challenge even beyond their accuracy. Recent advances in AI increasingly emphasize the necessity of integrating explainability into these systems. While most traditional AI methods and expert systems are inherently interpretable, the recent literature has focused primarily on explainability techniques for more complex models such as deep learning. This scoping review critically analyzes the existing literature regarding the explainability and interpretability of AI methods within the clinical domain. It offers a comprehensive overview of past and current research trends with the objective of identifying limitations that hinder the advancement of Explainable Artificial Intelligence (XAI) in the field of medicine. Such constraints encompass the diverse requirements of key stakeholders, including clinicians, patients, and developers, as well as cognitive barriers to knowledge acquisition, the absence of standardised evaluation criteria, the potential for mistaking explanations for causal relationships, and the apparent trade-off between model accuracy and interpretability. Furthermore, this review discusses possible research directions aimed at surmounting these challenges. These include alternative approaches to leveraging medical expertise to enhance interpretability within clinical settings, such as data fusion techniques and interdisciplinary assessments throughout the development process, emphasizing the relevance of taking into account the needs of final users to design trustable explainability methods.

show abstract

Section: Controversy Around Current Xai Methodsmentioning

confidence: 99%

A Scoping Review on the Progress, Applicability, and Future of Explainable Artificial Intelligence in Medicine

González-Alday,

García-Cuesta,

Kulikowski

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Interpretability is a key factor in afecting the attitudes of clinicians and patients toward CDSS based on AI [34,63]. Four themes emerge from the reviewed articles: (1) interpretable data type, (2) biomarkers, (3) interface for human-AI interaction, and (4) needs of clinicians and patients for interpretability.…”

Section: Interpretability Of Cdss From the Medical Perspectivementioning

confidence: 99%

Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

Xie

Liao

et al. 2023

Journal of Healthcare Engineering

View full text Add to dashboard Cite

Background. Artificial intelligence (AI) has developed rapidly, and its application extends to clinical decision support system (CDSS) for improving healthcare quality. However, the interpretability of AI-driven CDSS poses significant challenges to widespread application. Objective. This study is a review of the knowledge-based and data-based CDSS literature regarding interpretability in health care. It highlights the relevance of interpretability for CDSS and the area for improvement from technological and medical perspectives. Methods. A systematic search was conducted on the interpretability-related literature published from 2011 to 2020 and indexed in the five databases: Web of Science, PubMed, ScienceDirect, Cochrane, and Scopus. Journal articles that focus on the interpretability of CDSS were included for analysis. Experienced researchers also participated in manually reviewing the selected articles for inclusion/exclusion and categorization. Results. Based on the inclusion and exclusion criteria, 20 articles from 16 journals were finally selected for this review. Interpretability, which means a transparent structure of the model, a clear relationship between input and output, and explainability of artificial intelligence algorithms, is essential for CDSS application in the healthcare setting. Methods for improving the interpretability of CDSS include ante-hoc methods such as fuzzy logic, decision rules, logistic regression, decision trees for knowledge-based AI, and white box models, post hoc methods such as feature importance, sensitivity analysis, visualization, and activation maximization for black box models. A number of factors, such as data type, biomarkers, human-AI interaction, needs of clinicians, and patients, can affect the interpretability of CDSS. Conclusions. The review explores the meaning of the interpretability of CDSS and summarizes the current methods for improving interpretability from technological and medical perspectives. The results contribute to the understanding of the interpretability of CDSS based on AI in health care. Future studies should focus on establishing formalism for defining interpretability, identifying the properties of interpretability, and developing an appropriate and objective metric for interpretability; in addition, the user's demand for interpretability and how to express and provide explanations are also the directions for future research.

show abstract

A Reference Process for Assessing the Reliability of Predictive Analytics Results

Staudinger,

Schuetz,

Schrefl

2024

SN COMPUT. SCI.

View full text Add to dashboard Cite

Organizations employ data mining to discover patterns in historic data in order to learn predictive models. Depending on the predictive model the predictions may be more or less accurate, raising the question about the reliability of individual predictions. This paper proposes a reference process aligned with the CRISP-DM to enable the assessment of reliability of individual predictions obtained from a predictive model. The reference process describes activities along the different stages of the development process required to establish a reliability assessment approach for a predictive model. The paper then presents in more detail two specific approaches for reliability assessment: perturbation of input cases and local quality measures. Furthermore, this paper describes elements of a knowledge graph to capture important metadata about the development process and training data. The knowledge graph serves to properly configure and employ the reliability assessment approaches.

show abstract

How to Induce Trust in Medical AI Systems

Cited by 4 publications

References 12 publications

A Scoping Review on the Progress, Applicability, and Future of Explainable Artificial Intelligence in Medicine

A Scoping Review on the Progress, Applicability, and Future of Explainable Artificial Intelligence in Medicine

Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

A Reference Process for Assessing the Reliability of Predictive Analytics Results

Contact Info

Product

Resources

About