Interpretable &amp; Explorable Approximations of Black Box Models

Lakkaraju, Himabindu; Kamar, Ece; Caruana, Rich; Leskovec, Jure

doi:10.48550/arxiv.1707.01154

Cited by 54 publications

(76 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HCI research, and more broadly human-subject studies, are key to evaluating XAI in the context of use [30], identifying where it falls short, and informing human-centered solutions. While many studies showed positive results that XAI techniques can improve people's understanding of models [46,59,68,87], in this section we draw attention to a few pitfalls of XAI based on recent HCI research. 1 The difference between Why Not and How to Be That can be subtle and context-dependent.…”

Section: Pitfalls Of Xai: Minding the Gaps Between Algorithmic Explan...mentioning

confidence: 99%

Human-Centered Explainable AI (XAI): From Algorithms to User Experiences

Liao¹,

Varshney²

2021

Preprint

View full text Add to dashboard Cite

Book Chapter Draft 10/2021) As a technical sub-field of artificial intelligence (AI), explainable AI (XAI) has produced a vast collection of algorithms, providing a toolbox for researchers and practitioners to build XAI applications. With the rich application opportunities, explainability has moved beyond a demand by data scientists or researchers to comprehend the models they are developing, to become an essential requirement for people to trust and adopt AI deployed in numerous domains. However, explainability is an inherently human-centric property and the field is starting to embrace human-centered approaches. Human-computer interaction (HCI) research and user experience (UX) design in this area are becoming increasingly important. In this chapter, we begin with a high-level overview of the technical landscape of XAI algorithms, then selectively survey our own and other recent HCI works that take human-centered approaches to design, evaluate, provide conceptual and methodological tools for XAI. We ask the question "what are human-centered approaches doing for XAI " and highlight three roles that they play in shaping XAI technologies by helping navigate, assess and expand the XAI toolbox: to drive technical choices by users' explainability needs, to uncover pitfalls of existing XAI methods and inform new methods, and to provide conceptual frameworks for human-compatible XAI.

show abstract

Section: Pitfalls Of Xai: Minding the Gaps Between Algorithmic Explan...mentioning

confidence: 99%

Human-Centered Explainable AI (XAI): From Algorithms to User Experiences

Liao¹,

Varshney²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…While in recent years research on rule-based models has grown substantially, the problem of how to visualize the rules has gain COMPARATIVELY little attention. Decision rules are mostly presented as plain textual if-then statements, independently of whether the rules are used as model explanations [10], [28], [29], [30] or are used directly as interpretable predictive models [25], [26], [31]. For rules extracted from a decision tree [2], tree structures have been studied widely in the visualization research community (a gallery of tree visualization techniques can be found at treevis.net [32]) but only with little focus to structures designed to explore model behavior.…”

Section: Visual Representations Of Rules and Decision Treesmentioning

confidence: 99%

Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets

Yuan¹,

Barr²,

Overton³

et al. 2022

Preprint

View full text Add to dashboard Cite

One of the potential solutions for model interpretation is to train a surrogate model: a more transparent model that approximates the behavior of the model to be explained. Typically, classification rules or decision trees are used due to the intelligibility of their logic-based expressions. However, decision trees can grow too deep and rule sets can become too large to approximate a complex model. Unlike paths on a decision tree that must share ancestor nodes (conditions), rules are more flexible. However, the unstructured visual representation of rules makes it hard to make inferences across rules. To address these issues, we present a workflow that includes novel algorithmic and interactive solutions. First, we present Hierarchical Surrogate Rules (HSR), an algorithm that generates hierarchical rules based on user-defined parameters. We also contribute SURE, a visual analytics (VA) system that integrates HSR and interactive surrogate rule visualizations. Particularly, we present a novel feature-aligned tree to overcome the shortcomings of existing rule visualizations. We evaluate the algorithm in terms of parameter sensitivity, time performance, and comparison with surrogate decision trees and find that it scales reasonably well and outperforms decision trees in many respects. We also evaluate the visualization and the VA system by a usability study with 24 volunteers and an observational study with 7 domain experts. Our investigation shows that the participants can use feature-aligned trees to perform non-trivial tasks with very high accuracy. We also discuss many interesting observations that can be useful for future research on designing effective rule-based VA systems.

show abstract

“…Local-level feature explanations refers to the interpretations used to justify why the model made a specific decision for a single instance. Many existing approaches (Lundberg and Lee, 2017;Lakkaraju et al, 2017) can be adopted for local interpretations. Among the many existing explanation-generating methods, we adopted LIME (Ribeiro et al, 2016) as an example way for achieving local-level feature importance in the proposed framework.…”

Section: Local-level Explainable Feature Contributionsmentioning

confidence: 99%

When and Why does a Model Fail? A Human-in-the-loop Error Detection Framework for Sentiment Analysis

Liu,

Guo,

Mahmud

2021

Preprint

View full text Add to dashboard Cite

Although deep neural networks have been widely employed and proven effective in sentiment analysis tasks, it remains challenging for model developers to assess their models for erroneous predictions that might exist prior to deployment. Once deployed, emergent errors can be hard to identify in prediction runtime and impossible to trace back to their sources. To address such gaps, in this paper we propose an error detection framework for sentiment analysis based on explainable features. We perform global-level feature validation with human-in-the-loop assessment, followed by an integration of global and locallevel feature contribution analysis. Experimental results show that, given limited human-inthe-loop intervention, our method is able to identify erroneous model predictions on unseen data with high precision.

show abstract

Interpretable & Explorable Approximations of Black Box Models

Cited by 54 publications

References 6 publications

Human-Centered Explainable AI (XAI): From Algorithms to User Experiences

Human-Centered Explainable AI (XAI): From Algorithms to User Experiences

Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets

When and Why does a Model Fail? A Human-in-the-loop Error Detection Framework for Sentiment Analysis

Contact Info

Product

Resources

About