Faithful and Customizable Explanations of Black Box Models

Lakkaraju, Himabindu; Kamar, Ece; Caruana, Rich; Leskovec, Jure

doi:10.1145/3306618.3314229

Cited by 234 publications

(205 citation statements)

References 9 publications

Supporting

Mentioning

201

Contrasting

Unclassified

Order By: Relevance

“…The explanation model is g : ℝ d ′ → ℝ, g ∈ G , where G is a class of potentially interpretable models, such as linear models, decision trees, or rule lists; given a model g ∈ G , it can be visualized as an explanation to the human expert (for details please refer to (Ribeiro, Singh, & Guestrin, )). Another example for a posthoc system is black box explanations through transparent approximations (BETA), a model‐agnostic framework for explaining the behavior of any black‐box classifier by simultaneously optimizing for fidelity to the original model and interpretability of the explanation introduced by Lakkaraju, Kamar, Caruana, and Leskovec ().…”

Section: General Approaches Of Explainable Ai Modelsmentioning

confidence: 99%

Causability and explainability of artificial intelligence in medicine

Holzinger

Langs

Denk

et al. 2019

WIREs Data Min & Knowl

908

571

View full text Add to dashboard Cite

Explainable artificial intelligence (AI) is attracting much interest in medicine. Technically, the problem of explainability is as old as AI itself and classic AI represented comprehensible retraceable approaches. However, their weakness was in dealing with uncertainties of the real world. Through the introduction of probabilistic learning, applications became increasingly successful, but increasingly opaque. Explainable AI deals with the implementation of transparency and traceability of statistical black‐box machine learning methods, particularly deep learning (DL). We argue that there is a need to go beyond explainable AI. To reach a level of explainable medicine we need causability. In the same way that usability encompasses measurements for the quality of use, causability encompasses measurements for the quality of explanations. In this article, we provide some necessary definitions to discriminate between explainability and causability as well as a use‐case of DL interpretation and of human explanation in histopathology. The main contribution of this article is the notion of causability, which is differentiated from explainability in that causability is a property of a person, while explainability is a property of a system This article is categorized under: Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction

show abstract

Section: General Approaches Of Explainable Ai Modelsmentioning

confidence: 99%

Causability and explainability of artificial intelligence in medicine

Holzinger

Langs

Denk

et al. 2019

WIREs Data Min & Knowl

908

571

View full text Add to dashboard Cite

show abstract

“…One-off explanations are still the most popular operationalisation of explainability algorithms [34], where the explainer outputs a one-size-fits-all explanation in an attempt to make the behaviour of a predictive system transparent. A slight improvement over this scenario is to enable the explainer to account for user preferences when generating the explanations [22,29], but this modality is not common either.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Akula et al [1] presented a dialogue-driven explainability system that uses contrastive explanations based on predictions derived from And-Or graphs and handcrafted ontology, however generalising this technique may be challenging as it requires hand-crafting separate ontology and And-Or graph for each application. Lakkaraju et al [22] introduced rule-based explanations that the user can personalise by choosing which features will appear in the explanation-an off-line personalisation. Google published their what-if tool 5 which provides the explainee with an interactive interface that allows generating contrastive explanations of selected data points by modifying their features, i.e., asking "What if?"…”

Section: Background and Related Workmentioning

confidence: 99%

One Explanation Does Not Fit All

2020

View full text Add to dashboard Cite

The need for transparency of predictive systems based on Machine Learning algorithms arises as a consequence of their ever-increasing proliferation in the industry. Whenever black-box algorithmic predictions influence human affairs, the inner workings of these algorithms should be scrutinised and their decisions explained to the relevant stakeholders, including the system engineers, the system's operators and the individuals whose case is being decided. While a variety of interpretability and explainability methods is available, none of them is a panacea that can satisfy all diverse expectations and competing objectives that might be required by the parties involved. We address this challenge in this paper by discussing the promises of Interactive Machine Learning for improved transparency of black-box systems using the example of contrastive explanations-a state-of-the-art approach to Interpretable Machine Learning. Specifically, we show how to personalise counterfactual explanations by interactively adjusting their conditional statements and extract additional explanations by asking follow-up "What if?" questions. Our experience in building, deploying and presenting this type of system allowed us to list desired properties as well as potential limitations, which can be used to guide the development of interactive explainers. While customising the medium of interaction, i.e., the user interface comprising of various communication channels, may give an impression of personalisation, we argue that adjusting the explanation itself and its content is more important. To this end, properties such as breadth, scope, context, purpose and target of the explanation have to be considered, in addition to explicitly informing the explainee about its limitations and caveats. Furthermore, we discuss the challenges of mirroring the explainee's mental model, which is the main building block of intelligible human-machine interactions. We also deliberate on the risks of allowing the explainee to freely manipulate the explanations and thereby extracting information about the underlying predictive model, which might be leveraged by malicious actors to steal or game the model. Finally, building an end-to-end interactive explainability system is a challenging engineering task; unless the main goal is its deployment, we recommend "Wizard of Oz" studies as a proxy for testing and evaluating standalone interactive explainability algorithms.

show abstract

“…Building on the growing fear of AI systems creating a "black box society" [19], work in the area of explainable AI has explored ways in which difficult to understand machine learning models may be represented and communicated more simply to users [20]. For example, model understanding through space explanations enables translating an arbitrary black-box representation of a machine learning system into decision sets which capture the behavior of the black box in specific circumstances [21]. In interactive machine learning, users can view and correct classifications made by a system [22].…”

Section: )mentioning

confidence: 99%

Editable AI: Mixed Human-AI Authoring of Code Patterns

Chugh

Solis

LaToza

2019

2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)

View full text Add to dashboard Cite

Developers authoring HTML documents define elements following patterns which establish and reflect the visual structure of a document, such as making all images in a footer the same height by applying a class to each. To surface these patterns to developers and support developers in authoring consistent with these patterns, we propose a mixed human-AI technique for creating code patterns. Patterns are first learned from individual HTML documents through a decision tree, generating a representation which developers may view and edit. Code patterns are used to offer developers autocomplete suggestions, list examples, and flag violations. To evaluate our technique, we conducted a user study in which 24 participants wrote, edited, and corrected HTML documents. We found that our technique enabled developers to edit and correct documents more quickly and create, edit, and correct documents more successfully.

show abstract

Faithful and Customizable Explanations of Black Box Models

Cited by 234 publications

References 9 publications

Causability and explainability of artificial intelligence in medicine

Causability and explainability of artificial intelligence in medicine

One Explanation Does Not Fit All

Editable AI: Mixed Human-AI Authoring of Code Patterns

Contact Info

Product

Resources

About