Solving the Explainable AI Conundrum: How to Bridge the Gap Between Clinicians Needs and Developers Goals

Bienefeld, Nadine; Lüthy, Rahel; Brodbeck, Dominique; Boss, J. M.; Willms, Jan; Azzati, Jan; Blaser, Mirco; Keller, Emanuela

doi:10.21203/rs.3.rs-2326665/v1

Cited by 8 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Involving clinicians in the co-design of interpretable rather than fully transparent systems could thus be a solution to solving the explainable AI conundrum as a recent study has shown [44]. Given the high safety risks for patients, these considerations are particularly important for the tasks of "diagnostic decision-making", "prescribing medication or treatment", and "analyzing medical data".…”

Section: Discussionmentioning

confidence: 99%

Human-AI Teaming in Critical Care: A Comparative Analysis of Data Scientists’ and Clinicians’ Perspectives on AI Augmentation and Automation

Bienefeld,

Keller,

Grote

2024

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

Background: Artificial intelligence (AI) and machine learning hold immense potential for enhancing clinical and administrative healthcare tasks. However, slow adoption and implementation challenges highlight the need to consider how humans can effectively collaborate with AI within the broader socio-technical system.Objective: We aim to explore the optimal utilization of human and AI capabilities by determining suitable levels of human-AI teaming for safely and meaningfully augmenting or automating tasks. We focus on intensive care units (ICUs) as an example and provide recommendations for policymakers and healthcare practitioners regarding AI deployment in healthcare settings. Methods:We conducted a systematic task analysis in six ICUs in Europe and carried out an international Delphi survey involving 19 health data scientists from academia and industry (response rate = 95%; 21% female; mean age = 38.6 years; mean experience = 12.63). Consensus was reached on the appropriate level of human-AI teaming for each task (Level 1 = no performance benefits from AI; Level 2 = AI augments human performance; Level 3 = Human augments AI performance; Level 4 = AI performs without human input). Ethical and social implications, as well as control and accountability distribution, were also considered by experts.

show abstract

Section: Discussionmentioning

confidence: 99%

Human-AI Teaming in Critical Care: A Comparative Analysis of Data Scientists’ and Clinicians’ Perspectives on AI Augmentation and Automation

Bienefeld,

Keller,

Grote

2024

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

show abstract

“…Finally, we observed a strong preference among medical students for AI systems that are explainable rather than highly accurate. This mirrors the growing emphasis on 'Explainable AI' in the medical field and underscores the urgent need for developing AI algorithms that transparently disclose their decisions and limitations, especially when applied to the medical domain, to promote trust and acceptance among healthcare students, professionals, and patients [51][52][53][54].…”

Section: Discussionmentioning

confidence: 99%

Medical students’ perceptions towards artificial intelligence in education and practice: A multinational, multicenter cross-sectional study

Busch,

Hoffmann,

Truhn

et al. 2023

Preprint

View full text Add to dashboard Cite

BackgroundArtificial intelligence (AI) is anticipated to fundamentally change the educational and professional landscape for the next generation of physicians, but its successful integration depends on the global perspectives of all stakeholders. Previous medical student surveys were limited by small sample sizes or geographic constraints, hindering a global comparison of perceptions. This study aims to explore current medical students’ attitudes towards AI in medical education and the profession on a broad, international scale and to examine regional differences in perspectives.Methods and FindingsThis international multicenter cross-sectional study developed and validated an anonymous online survey of 15 multiple-choice items to assess medical, dentistry, and veterinary students’ AI knowledge and attitudes toward the utilization of AI in healthcare, the current state of AI education, and regional differences in perspectives. Between April and October 2023, 4,313 medical, 205 dentistry, and 78 veterinary students from 192 faculties in 48 countries responded to the survey (average response rate: 0.2%, standard deviation: 0.4%). Most participants studied in European countries (N=2,350), followed by North/South America (N=1,070) and Asia (N=944). Students expressed predominantly positive attitudes towards the use of AI in healthcare (67.6%, N=3,091) and the desire for more AI teaching in their curricula (76.1%, N=3,474). However, they reported limited general knowledge of AI (75.3%, N=3,451), the absence of AI-related courses (76.3%, N=3,497), and felt inadequately prepared to use AI in their future careers (57.9%, N=2,652). The subgroup analyses revealed regional differences in perceptions, although predominantly with small effect sizes. The main limitations include the low response rate per institution, which was calculated on total enrollment across all degree programs, and the risk of selection bias.ConclusionsThis study highlights the favorable perceptions of international medical students towards incorporating AI in healthcare practice while emphasizing the importance of integrating AI teaching into medical education.Graphical abstract

show abstract

“…This study might produce explanations that continue to be correct, pertinent, and in line with how drug development is evolving [87]. The insights provided to researchers and clinicians can maintain their interpretability and reliability by using adaptable XAI frameworks that automatically update explanations in response to model updates or changes in data distribution [88]. This will enable informed decision-making even in the face of changing complexity.…”

Section: B Future Research Directions 1) Dynamic Explanation Adaptationmentioning

confidence: 95%

Explainable Artificial Intelligence for Drug Discovery and Development: A Comprehensive Survey

Alizadehsani,

Oyelere,

Hussain

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The field of drug discovery has experienced a remarkable transformation with the advent of artificial intelligence (AI) and machine learning (ML) technologies. However, as these AI and ML models are becoming more complex, there is a growing need for transparency and interpretability of the models. Explainable Artificial Intelligence (XAI) is a novel approach that addresses this issue and provides a more interpretable understanding of the predictions made by machine learning models. In recent years, there has been an increasing interest in the application of XAI techniques to drug discovery. This review article provides a comprehensive overview of the current state-of-the-art in XAI for drug discovery, including various XAI methods, their application in drug discovery, and the challenges and limitations of XAI techniques in drug discovery. The article also covers the application of XAI in drug discovery, including target identification, compound design, and toxicity prediction. Furthermore, the article suggests potential future research directions for the application of XAI in drug discovery. This review article aims to provide a comprehensive understanding of the current state of XAI in drug discovery and its potential to transform the field.

show abstract

Solving the Explainable AI Conundrum: How to Bridge the Gap Between Clinicians Needs and Developers Goals

Cited by 8 publications

References 18 publications

Human-AI Teaming in Critical Care: A Comparative Analysis of Data Scientists’ and Clinicians’ Perspectives on AI Augmentation and Automation

Human-AI Teaming in Critical Care: A Comparative Analysis of Data Scientists’ and Clinicians’ Perspectives on AI Augmentation and Automation

Medical students’ perceptions towards artificial intelligence in education and practice: A multinational, multicenter cross-sectional study

Explainable Artificial Intelligence for Drug Discovery and Development: A Comprehensive Survey

Contact Info

Product

Resources

About