LEAFAGE: Example-based and Feature importance-based Explanations for Black-box ML models

Abstract

Explainable Artificial Intelligence (XAI) is an emergent research field which tries to cope with the lack of transparency of AI systems, by providing human understandable explanations for the underlying Machine Learning models. This work presents a new explanation extraction method called LEAFAGE. Explanations are provided both in terms of feature importance and of similar classification examples. The latter is a well known strategy for problem solving and justification in social science. LEAFAGE leverages on the fact that the reasoning behind a single decision/prediction for a single data point is generally simpler to understand than the complete model; it produces explanations by generating simpler yet locally accurate approximations of the original model. LEAFAGE performs overall better than the current state of the art in terms of fidelity of the model approximation, in particular when Machine Learning models with non-linear decision boundaries are analysed. LEAFAGE was also tested in terms of usefulness for the user, an aspect still largely overlooked in the scientific literature. Results show interesting and partly counter-intuitive findings, such as the fact that providing no explanation is sometimes better than providing certain kinds of explanation.

Intuitively Assessing ML Model Reliability through Example-Based Explanations and Editing Model Inputs

“…Others try to produce explanations more grounded in the features of the data, such as a ranking of features important for the prediction or a decision-tree approximating the model's logic [16,33,46]. However, a growing body of work that has tried to empirically measure the efficacy of many of these methods has shown that they often do not actually affect or improve human decision-making [2,8,30,43], and in practice are primarily used for internal model debugging [6].…”

An exact counterfactual-example-based approach to tree-ensemble models interpretability

“…2) CF model agnostic interpretability methods: Among other model-agnostic approaches that were tested with reported results on tree ensemble models, we can mention the LORE [23], LEAFAGE [24], and CLEAR [25] approaches. The LORE approach uses local interpretable surrogates in order to derive sets of CF rules.…”

Explainable artificial intelligence in forensics: Realistic explanations for number of contributor predictions of DNA profiles

“…It was important to evaluate the visualization from the perspective of the end-users, an aspect often brushed over in XAI studies [56,59]. The explanation was specifically designed for DNA experts within the context of NOC estimation, so we invited DNA experts from the NFI to participate in a user study.…”