Explaining Classifiers using Adversarial Perturbations on the Perceptual Ball

Elliott, Andrew; Law, S.S.; Russell, Chris

doi:10.1109/cvpr46437.2021.01055

Cited by 15 publications

(15 citation statements)

References 23 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, during the test phase (on images not seen at the training time) the explanation is not provided since we aim to measure how well a user is able to predict the model and not the ability of the explanation to leak the label. and the field of Explainable AI is no exception [18,20,25,36,43,51]. We use this dataset because we expect it to be representative of real-world scenarios where it is difficult to understand what the model is relying on for its decisions.…”

Section: Methodsmentioning

confidence: 99%

“…Evaluations based on ground-truth annotations A first class of evaluation approaches scores explainability methods according to their ability to identify image locations that overlap with the target object defined either by a humanderived bounding box or a segmentation mask [17,19,20,37,41,51]. More recently, the evaluation method called Pointing Game [56] counts the number of times the most important region according to the explanation intersects with the location of the object to classify.…”

Section: Related Workmentioning

confidence: 99%

“…Despite the existence of a wide range of explainability methods, assessing the quality and reliability of these methods remains an open problem. The community has taken up the issue and proposed two commonly used approaches to evaluate methods: one based on ground truth annotations [17,19,20,37,41,51,56] and a second more commonly used on faithfulness metrics [19,20,29,37,40,51].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods

Fel¹,

Colin²,

Cadène³

et al. 2021

Preprint

View full text Add to dashboard Cite

A multitude of explainability methods and theoretical evaluation scores have been proposed. However, it is not yet known: (1) how useful these methods are in real-world scenarios and (2) how well theoretical measures predict the usefulness of these methods for practical use by a human. To fill this gap, we conducted human psychophysics experiments at scale to evaluate the ability of human participants (n = 1, 150) to leverage representative attribution methods to learn to predict the decision of different image classifiers. Our results demonstrate that theoretical measures used to score explainability methods poorly reflect the practical usefulness of individual attribution methods in real-world scenarios. Furthermore, the degree to which individual attribution methods helped human participants predict classifiers' decisions varied widely across categorization tasks and datasets.Overall, our results highlight fundamental challenges for the field -suggesting a critical need to develop better explainability methods and to deploy human-centered evaluation approaches. We will make the code of our framework available to ease the systematic evaluation of novel explainability methods.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods

Fel¹,

Colin²,

Cadène³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Adversarial Examples as Explanation-Rather than using a perturbation function and optimizing a mask, [48,12] propose to find for each input image an L 2 -close adversarial example (to be compared to the input) that impacts the classifier's decision within a constrained space. More recently, [8] train two models with the same architecture and partly shared parameters to produce for a given input image, L 1 ,L 2 -close images with respectively similar and opposite classifications compared to the original one.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, the contribution of the stable image is less important than what is described in[8], even if it slightly improves the localization (see tables 2a and 2b) SyCE w/o St. vs SyCE). This is due to a generation process which is not penalized with L p norms as in[8,12]. Finally, although CyCE is the best performer for domain translation (see section F.1) and is competitive with other works of the literature, it obtains poorer localization results than SyCE.…”

mentioning

confidence: 99%

Leveraging Conditional Generative Models in a General Explanation Framework of Classifier Decisions

Charachon¹,

Cournède²,

Hudelot³

et al. 2021

Preprint

View full text Add to dashboard Cite

Providing a human-understandable explanation of classifiers' decisions has become imperative to generate trust in their use for dayto-day tasks. Although many works have addressed this problem by generating visual explanation maps, they often provide noisy and inaccurate results forcing the use of heuristic regularization unrelated to the classifier in question. In this paper, we propose a new general perspective of the visual explanation problem overcoming these limitations. We show that visual explanation can be produced as the difference between two generated images obtained via two specific conditional generative models. Both generative models are trained using the classifier to explain and a database to enforce the following properties: (i) All images generated by the first generator are classified similarly to the input image, whereas the second generator's outputs are classified oppositely. (ii) Generated images belong to the distribution of real images. (iii) The distances between the input image and the corresponding generated images are minimal so that the difference between the generated elements only reveals relevant information for the studied classifier. Using symmetrical and cyclic constraints, we present two different approximations and implementations of the general formulation. Experimentally, we demonstrate significant improvements w.r.t the state-of-the-art on three different public data sets. In particular, the localization of regions influencing the classifier is consistent with human annotations.

show abstract