Generating Counterfactual and Contrastive Explanations using SHAP

Rathi, Shubham

doi:10.48550/arxiv.1906.09293

Cited by 17 publications

(24 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, in an autonomous car it is not practical to go through all alternative angles a steering wheel could have been turned to observe alternative results. Deriving the foil from the context is part of the explanation facility's task, but apart from some attempts in IML [235,208,185,181], has not been widely discussed in the context of RL. While not the focus of their research some work in dynamic programming, such as Erwig et al [74], have found that the context for contrastive explanations could be anticipated by identifying principal and minor categories and using these to anticipate user questions through value decomposition.…”

Section: Results Of Xrl-behaviourmentioning

confidence: 99%

Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey

Dazeley¹,

Vamplew²,

Cruz³

2021

Preprint

View full text Add to dashboard Cite

Broad Explainable Artificial Intelligence (Broad-XAI) moves away from interpreting individual decisions based on a single datum and aims to provide integrated explanations from multiple machine learning algorithms into a coherent explanation of an agent's behaviour that is aligned to the communication needs of the explainee. Reinforcement Learning (RL) methods, we propose, provide a potential backbone for the cognitive model required for the development of Broad-XAI. RL represents a suite of approaches that have had increasing success in solving a range of sequential decision-making problems. However, these algorithms all operate as black-box problem solvers, where they obfuscate their decision-making policy through a complex array of values and functions. EXplainable RL (XRL) is relatively recent field of research that aims to develop techniques to extract concepts from the agent's: perception of the environment; intrinsic/extrinsic motivations/beliefs; Q-values, goals and objectives. This paper aims to introduce a conceptual framework, called the Causal XRL Framework (CXF), that unifies the current XRL research and uses RL as a backbone to the development of Broad-XAI. Additionally, we recognise that RL methods have the ability to incorporate a range of technologies to allow agents to adapt to their environment. CXF is designed for the incorporation of many standard RL extensions and integrated with external ontologies and communication facilities so that the agent can answer questions that explain outcomes and justify its decisions. This paper aims to: establish XRL as a distinct branch of Explainable Artificial Intelligence (XAI); introduce a conceptual framework for XRL; review existing approaches explaining agent behaviour; and identify opportunities for future research. Finally, this paper will discuss how additional information can be extracted and ultimately integrated into models of communication, facilitating the development of Broad-XAI.

show abstract

Section: Results Of Xrl-behaviourmentioning

confidence: 99%

Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey

Dazeley¹,

Vamplew²,

Cruz³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Therefore, datasets of various data types (both categorical, numerical and mixed variables), application domain, number of features, ratio of feature types, class balance and kinds of data constraints are selected (see Table 2). Several of these datasets are widely used in counterfactual explanation studies like Adult [53,29,64,74,21,76,10,45], Statlog -German Credit [53,21,59,52], Breast Cancer Wisconsin (BCW) [74,5,76,4,39] and Wine [60,42,4]. Note that the number of datasets included in the papers where these algorithms have been proposed range from 1 to only 4 [53], which further motivates the need for a large-scale benchmarking study.…”

Section: Datasetsmentioning

confidence: 99%

A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data

Mazzine,

Martens

2021

Preprint

View full text Add to dashboard Cite

Counterfactual explanations are viewed as an effective way to explain machine learning predictions. This interest is reflected by a relatively young literature with already dozens of algorithms aiming to generate such explanations. These algorithms are focused on finding how features can be modified to change the output classification. However, this rather general objective can be achieved in different ways, which brings about the need for a methodology to test and benchmark these algorithms. The contributions of this work are manifold: First, a large benchmarking study of 10 algorithmic approaches on 22 tabular datasets is performed, using 9 relevant evaluation metrics. Second, the introduction of a novel, first of its kind, framework to test counterfactual generation algorithms. Third, a set of objective metrics to evaluate and compare counterfactual results. And finally, insight from the benchmarking results that indicate which approaches obtain the best performance on what type of dataset. This benchmarking study and framework can help practitioners in determining which technique and building blocks most suit their context, and can help researchers in the design and evaluation of current and future counterfactual generation algorithms. Our findings show that, overall, there's no single best algorithm to generate counterfactual explanations as the performance highly depends on properties related to the dataset, model, score and factual point specificities.

show abstract

“…They are attractive since they ignore the underline structure of the target model, allowing a broader spectrum of applications. In some cases disclosing the system's inner mechanism can make it vulnerable to attacks or gamification [46]. Systems gamification may be useful in certain circumstances [47], but counterproductive in others [48].…”

Section: What and How To Explain?mentioning

confidence: 99%

Towards an Explanation Space to Align Humans and Explainable-AI Teamwork

Cabour,

Morales,

Ledoux

et al. 2021

Preprint

View full text Add to dashboard Cite

Providing meaningful and actionable explanations to end-users is a fundamental prerequisite for implementing explainable intelligent systems in the real world. Explainability is a situated interaction between a user and the AI system rather than being static design principles. The content of explanations is context-dependent and must be defined by evidence about the user and its context. This paper seeks to operationalize this concept by proposing a formative architecture that defines the explanation space from a user-inspired perspective. The architecture comprises five intertwined components to outline explanation requirements for a task: (1) the end-users mental models, (2) the end-users cognitive process, (3) the user interface, (4) the human-explainer agent, and the (5) agent process. We first define each component of the architecture. Then we present the Abstracted Explanation Space, a modeling tool that aggregates the architecture's components to support designers in systematically aligning explanations with the end-users work practices, needs, and goals. It guides the specifications of what needs to be explained (content -end-users mental model), why this explanation is necessary (context -end-users cognitive process), to delimit how to explain it (format -human-explainer agent and user interface), and when should the explanations be given. We then exemplify the tool's use in an ongoing case study in the aircraft maintenance domain. Finally, we discuss possible contributions of the tool, known limitations/areas for improvement, and future work to be done.

show abstract

Generating Counterfactual and Contrastive Explanations using SHAP

Cited by 17 publications

References 2 publications

Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey

Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey

A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data

Towards an Explanation Space to Align Humans and Explainable-AI Teamwork

Contact Info

Product

Resources

About