Convolutional Dynamic Alignment Networks for Interpretable Classifications

Towards Better Understanding Attribution Methods

“…We present our evaluation settings for better understanding the strengths and shortcomings of attribution methods. Similar to the Grid Pointing Game (GridPG) [4], these metrics evaluate attribution methods on image grids with multiple classes. In particular, we propose a novel quantitative metric, DiFull, and extension to it, DiPart (3.1), as stricter tests of model faithfulness than GridPG.…”

“…[25]. Therefore, recent work [3,4,24] proposes creating a grid of inputs from distinct classes and measuring localization to the entire grid cell, which allows evaluation on datasets where bounding boxes are not available. However, this does not guarantee that the model only uses information from within the grid cell, and may fail for similar looking features (Fig.…”

“…In the following, we introduce the quantitative metrics that we use to compare attribution methods. For this, we first describe GridPG and the grid dataset construction it uses [4]. We then devise a novel setting, in which we carefully control which features can influence the model output.…”

“…For GridPG [4], the attribution methods are evaluated on a synthetic grid of n × n images in which each class may occur at most once. In particular, for each of the occurring classes, GridPG measures the fraction of positive attribution assigned to the respective grid cell versus the overall amount of positive attribution.…”

“…1, right), which complements the evaluation on individual samples. (4) We propose a post-processing smoothing step that significantly improves localization performance on some attribution methods. We observe significant differences when evaluating these smoothed attributions on different architectures, which highlights how architectural design choices can influence an attribution method's applicability.…”