Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation

Abstract

With the wide use of deep neural networks (DNN), model interpretability has become a critical concern, since explainable decisions are preferred in high-stake scenarios. Current interpretation techniques mainly focus on the feature attribution perspective, which are limited in indicating why and how particular explanations are related to the prediction. To this end, an intriguing class of explanations, named counterfactuals, has been developed to further explore the "what-if" circumstances for interpretation, and enables the reasoning capability on black-box models. However, generating counterfactuals for raw data instances (i.e., text and image) is still in the early stage due to its challenges on high data dimensionality and unsemantic raw features. In this paper, we design a framework to generate counterfactuals specifically for raw data instances with the proposed Attribute-Informed Perturbation (AIP). By utilizing generative models conditioned with different attributes, counterfactuals with desired labels can be obtained effectively and efficiently. Instead of directly modifying instances in the data space, we iteratively optimize the constructed attribute-informed latent space, where features are more robust and semantic. Experimental results on real-world texts and images demonstrate the effectiveness, sample quality as well as efficiency of our designed framework, and show the superiority over other alternatives. Besides, we also introduce some practical applications based on our framework, indicating its potential beyond the model interpretability aspect.

Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations

“…The first family of methods conditions the generative model on attributes, by e.g. using a conditional GAN [26,33,50,59,60]. This dependency on attribute information can restrict the applicability of these methods in scenarios where annotations are scarce.…”

On the Importance of Domain-specific Explanations in AI-based Cybersecurity Systems (Technical Report)

“…This is not a popular feature in the literature, likely because it is not an easy task to produce appropriate comparisons. Counterfactual approaches (such as [14,34,28]) are a particular case where the contrast is based on synthetic situations (values of features/attributes); another example is that of balanced explanations, which appear in [10].…”

Counterfactual Explanations for Models of Code

“…These techniques are very related to the previously discussed perturbationbased mechanisms, but they come with a stronger guarantee, namely that the perturbations are guaranteed to change the model's prediction. The generation of counterfactual explanations has received significant attention in the NLP community [22,23,36,45,46]. In the simplest case, these counterfactuals can be generated by deleting words from the input text [23] or via rewrite-rules such as adding negations or shuffling words [45].…”