DARPA’s Explainable AI (XAI) program: A retrospective

Gunning, Dave; Vorm, Eric S.; Wang, Yunyan; Turek, Matt

doi:10.22541/au.163699841.19031727/v1

Cited by 24 publications

(15 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, “Explainable AI (XAI)” has become a popular research area. The Defense Advanced Research Projects Agency (DARPA) launched the XAI program in 2017, intending to create new or modified ML technologies and produce explainable models that enable users to understand, trust, and effectively manage AI systems [ 78 ]. In 2018, the General Data Protection Regulation (GDPR) of the European Union also stated that data subjects have a right to request explanations about automated decisions made by algorithms [ 79 ].…”

Section: Discussionmentioning

confidence: 99%

Characteristics of Artificial Intelligence Clinical Trials in the Field of Healthcare: A Cross-Sectional Study on ClinicalTrials.gov

Wang

Xiu

Liu

et al. 2022

IJERPH

View full text Add to dashboard Cite

Artificial intelligence (AI) has driven innovative transformation in healthcare service patterns, despite a lack of understanding of its performance in clinical practice. We conducted a cross-sectional analysis of AI-related trials in healthcare based on ClinicalTrials.gov, intending to investigate the trial characteristics and AI’s development status. Additionally, the Neo4j graph database and visualization technology were employed to construct an AI technology application graph, achieving a visual representation and analysis of research hotspots in healthcare AI. A total of 1725 eligible trials that were registered in ClinicalTrials.gov up to 31 March 2022 were included in this study. The number of trial registrations has dramatically grown each year since 2016. However, the AI-related trials had some design drawbacks and problems with poor-quality result reporting. The proportion of trials with prospective and randomized designs was insufficient, and most studies did not report results upon completion. Currently, most healthcare AI application studies are based on data-driven learning algorithms, covering various disease areas and healthcare scenarios. As few studies have publicly reported results on ClinicalTrials.gov, there is not enough evidence to support an assessment of AI’s actual performance. The widespread implementation of AI technology in healthcare still faces many challenges and requires more high-quality prospective clinical validation.

show abstract

Section: Discussionmentioning

confidence: 99%

Characteristics of Artificial Intelligence Clinical Trials in the Field of Healthcare: A Cross-Sectional Study on ClinicalTrials.gov

Wang

Xiu

Liu

et al. 2022

IJERPH

View full text Add to dashboard Cite

show abstract

“…As a consequence, with the advent of DL, there are increasing discussions in the field about the relationship between ML model complexity and interpretability and the tendency to use models that are too complicated for prediction tasks at hand ( Rudin, 2019 ). Furthermore, increasing attention is paid to explainable ML ( Belle and Papantonis, 2021 ; Rodríguez-Pérez and Bajorath, 2021a ) and the overarching area of explainable AI (XAI) ( Gunning et al., 2019 , 2021 ; Jiménez-Luna et al., 2020 ; Xu et al., 2019 ). XAI refers to different categories of computational approaches for rationalizing ML models and their decisions in different areas of basic and applied research ( Gunning et al., 2019 ; Jiménez-Luna et al., 2020 ; Xu et al., 2019 ) as well as in scientific teaching ( Clancey and Hoffman, 2021 ).…”

Section: Introductionmentioning

confidence: 99%

“…Explanation methods are equally relevant for classification and regression models ( Letzgus et al., 2021 ; Rodríguez-Pérez and Bajorath, 2020a ). Conceptually different XAI approaches include methods for feature weighting or attribution, causal methods, counterfactuals and contrastive explanations, transparent probabilistic models, or graph convolution analysis methods ( Gunning et al., 2021 ; Jiménez-Luna et al., 2020 ). In addition, local approximation models have been introduced for instance-based explanations of decisions by complex black box models ( Ribeiro et al., 2016 ; Lundberg and Lee, 2017 ).…”

Section: Introductionmentioning

confidence: 99%

EdgeSHAPer: Bond-centric Shapley value-based explanation method for graph neural networks

et al. 2022

View full text Add to dashboard Cite

“…The need to transform black-box decisions into transparent decisions for human decision-makers has led to a new field of study known as explainable AI (XAI) 1 [11]. Many XAI techniques are found in the literature to make ML systems explainable.…”

Section: Introductionmentioning

confidence: 99%

FCE: Feedback Based Counterfactual Explanations for Explainable AI

et al. 2022

View full text Add to dashboard Cite

Artificial Intelligence provides accurate predictions for critical applications (e.g., healthcare, finance), but lacks the ability to explain its internal mechanism in most applications which require high interaction with humans. Even if many studies analyze machine learning models and their learning behaviour and eventually provide an interpretation of the inner mechanics of these models, these studies often entail a simpler surrogate model generate explanations by producing a piece of interpretable information such as feature scores. The crucial caveat against these studies is the lack of human involvement in the design and evaluation of explanations, consequently giving rise to trust issues and lack of acceptance and understanding. To this end, we address this limitation by involving humans in the counterfactual explanation generation process which is enriched with user feedback and thus enhancing the automated explanations which are better aligned with user expectations. In this paper, we propose a user feedback based counterfactual explanation approach for explainable Artificial Intelligence. In our work, we utilize feedback in two ways: first, feedback to customize the explanations locally that helps in providing the neighbourhood to discern the feasible explanations; and second, to evaluate the generated explanations.INDEX TERMS Counterfactual explanations, Explainable AI, Human-in-the-loop, Interactive machine learning, User feedback.

show abstract

DARPA’s Explainable AI (XAI) program: A retrospective

Cited by 24 publications

References 6 publications

Characteristics of Artificial Intelligence Clinical Trials in the Field of Healthcare: A Cross-Sectional Study on ClinicalTrials.gov

Characteristics of Artificial Intelligence Clinical Trials in the Field of Healthcare: A Cross-Sectional Study on ClinicalTrials.gov

EdgeSHAPer: Bond-centric Shapley value-based explanation method for graph neural networks

FCE: Feedback Based Counterfactual Explanations for Explainable AI

Contact Info

Product

Resources

About