Explainable image classification with evidence counterfactual

Abstract

The complexity of state-of-the-art modeling techniques for image classification impedes the ability to explain model predictions in an interpretable way. A counterfactual explanation highlights the parts of an image which, when removed, would change the predicted class. Both legal scholars and data scientists are increasingly turning to counterfactual explanations as these provide a high degree of human interpretability, reveal what minimal information needs to be changed in order to come to a different prediction and do not require the prediction model to be disclosed. Our literature review shows that existing counterfactual methods for image classification have strong requirements regarding access to the training data and the model internals, which often are unrealistic. Therefore, SEDC is introduced as a model-agnostic instance-level explanation method for image classification that does not need access to the training data. As image classification tasks are typically multiclass problems, an additional contribution is the introduction of the SEDC-T method that allows specifying a target counterfactual class. These methods are experimentally tested on ImageNet data, and with concrete examples, we illustrate how the resulting explanations can give insights in model decisions. Moreover, SEDC is benchmarked against existing model-agnostic explanation methods, demonstrating stability of results, computational efficiency and the counterfactual nature of the explanations.

Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoning

“…Explainable AI has been an important topic in recommender systems [5,6,13,36,41,46,47], natural language processing [8,16,20] and computer vision [7,10,15,25,38]. To improve the transparency of deep neural networks, many explanation techniques have been proposed in recent years.…”

The Reality of High Performing Deep Learning Models: A Case Study on Document Image Classification

“…However, this work was based solely on textual data, in contrast to our approach which focuses on image data classification. Vermeire et al (2020) [31] extended the idea of SEDC to the visual domain to generate counterfactual explanations for image classification tasks. Recent work in this area has focused on generating realistic counterfactual explanations using Generative Adversarial Nets (GANs) [32], [33].…”

“…To find out which document features were the most important to the models, we used an approach similar to [31] to generate counterfactual explanations. However, instead of a searchbased perturbation, we used the feature importance map generated by DeepSHAP [15] as the basis for the feature perturbation to generate the counterfactual explanations.…”

“…We now describe the algorithm and how it can be used to find the features considered most relevant by the model for a given prediction. Given an image input x i , the true class label y i , and the model f (x), the counterfactual explanation c i is defined according to [31] as a subset of the features of x i where removing them leads to a class change and can be formulated by the following equations:…”

“…Although we have formulated the main idea of the approach, we still need to discuss the feature removal process to generate the counterfactual explanation c i . Vermeire et al (2020) [31] used a quick-shift algorithm to segment the images, and then applied a search-based algorithm over the possible set of perturbations that lead to the largest reduction in confidence of the target class. We used a different approach where we first used DeepSHAP to create a pixel-wise feature importance matrix e i and then segmented the feature importance map using a grid of box size G × G. Then, we computed the total importance over each grid box as the sum of the SHAP values within the box.…”