Diagnosing AI Explanation Methods with Folk Concepts of Behavior

Jacovi, Alon; Bastings, Jasmijn; Gehrmann, Sebastian; Goldberg, Yoav; Filippova, Katja

doi:10.48550/arxiv.2201.11239

Cited by 2 publications

(1 citation statement)

References 81 publications

(111 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ϕ f should be continuous. This robustness can be seen as a measure of coherence [18]. his type of robustness appears to be difficult to achieve, however, because a sequence of empirical counterexamples have been found in which several ethods like LIME, SHAP and Integrated Gradients are not robust [14,37,10,3].…”

Section: Attribution Methods and Counterfactual Explanationsmentioning

confidence: 99%

Attribution-based Explanations that Provide Recourse Cannot be Robust

Fokkema¹,

Heide²,

Erven³

2022

Preprint

View full text Add to dashboard Cite

Different users of machine learning methods require different explanations, depending on their goals. To make machine learning accountable to society, one important goal is to get actionable options for recourse, which allow an affected user to change the decision f (x) of a machine learning system by making limited changes to its input x. We formalize this by providing a general definition of recourse sensitivity, which needs to be instantiated with a utility function that describes which changes to the decisions are relevant to the user. This definition applies to local attribution methods, which attribute an importance weight to each input feature. It is often argued that such local attributions should be robust, in the sense that a small change in the input x that is being explained, should not cause a large change in the feature weights. However, we prove formally that it is in general impossible for any single attribution method to be both recourse sensitive and robust at the same time. It follows that there must always exist counterexamples to at least one of these properties. We provide such counterexamples for several popular attribution methods, including LIME, SHAP, Integrated Gradients and SmoothGrad. Our results also cover counterfactual explanations, which may be viewed as attributions that describe a perturbation of x. We further discuss possible ways to work around our impossibility result, for instance by allowing the output to consist of sets with multiple attributions. Finally, we strengthen our impossibility result for the restricted case where users are only able to change a single attribute of x, by providing an exact characterization of the functions f to which impossibility applies.

show abstract

Section: Attribution Methods and Counterfactual Explanationsmentioning

confidence: 99%

Attribution-based Explanations that Provide Recourse Cannot be Robust

Fokkema¹,

Heide²,

Erven³

2022

Preprint

View full text Add to dashboard Cite

show abstract

XAINES: Explaining AI with Narratives

Hartmann

Feldhus

et al. 2022

Künstl Intell

View full text Add to dashboard Cite

Artificial Intelligence (AI) systems are increasingly pervasive: Internet of Things, in-car intelligent devices, robots, and virtual assistants, and their large-scale adoption makes it necessary to explain their behaviour, for example to their users who are impacted by their decisions, or to their developers who need to ensure their functionality. This requires, on the one hand, to obtain an accurate representation of the chain of events that caused the system to behave in a certain way (e.g., to make a specific decision). On the other hand, this causal chain needs to be communicated to the users depending on their needs and expectations. In this phase of explanation delivery, allowing interaction between user and model has the potential to improve both model quality and user experience. The XAINES project investigates the explanation of AI systems through narratives targeted to the needs of a specific audience, focusing on two important aspects that are crucial for enabling successful explanation: generating and selecting appropriate explanation content, i.e. the information to be contained in the explanation, and delivering this information to the user in an appropriate way. In this article, we present the project’s roadmap towards enabling the explanation of AI with narratives.

show abstract

Diagnosing AI Explanation Methods with Folk Concepts of Behavior

Cited by 2 publications

References 81 publications

Attribution-based Explanations that Provide Recourse Cannot be Robust

Attribution-based Explanations that Provide Recourse Cannot be Robust

XAINES: Explaining AI with Narratives

Contact Info

Product

Resources

About