SWAG: Superpixels Weighted by Average Gradients for Explanations of CNNs

Abstract

Providing an explanation of the operation of CNNs that is both accurate and interpretable is becoming essential in fields like medical image analysis, surveillance, and autonomous driving. In these areas, it is important to have confidence that the CNN is working as expected and explanations from saliency maps provide an efficient way of doing this. In this paper, we propose a pair of complementary contributions that improve upon the state of the art for region-based explanations in both accuracy and utility. The first is SWAG, a method for generating accurate explanations quickly using superpixels for discriminative regions which is meant to be a more accurate, efficient, and tunable drop in replacement method for Grad-CAM, LIME, or other region-based methods. The second contribution is based on an investigation into how to best generate the superpixels used to represent the features found within the image. Using SWAG, we compare using superpixels created from the image, a combination of the image and backpropagated gradients, and the gradients themselves. To the best of our knowledge, this is the first method proposed to generate explanations using superpixels explicitly created to represent the discriminative features important to the network. To compare we use both ImageNet and challenging fine-grained datasets over a range of metrics. We demonstrate experimentally that our methods provide the best local and global accuracy compared to Grad-CAM, Grad-CAM++, LIME, XRAI, and RISE.

SWAG-V: Explanations for Video using Superpixels Weighted by Average Gradients

“…Recently, a set of techniques have been introduced that aim to pool the individual pixel scores given by gradientbased techniques into more interpretable regions. These are XRAI [13] and SWAG [10]. Both have a similar approach in that they generate superpixels using the input image, and then assign a score to these regions based on the gradient values that lie within them.…”

“…However, assigning a score for individual pixels often has the appearance of noise and has been deemed to be less interpretable than methods which score larger regions [13,29,30]. Recently techniques have been introduced that pool these individual pixel scores into more interpretable regions [10,13]. Of these, Superpixels Weighted by Average Gradient (SWAG) [10], is able to produce explanations efficiently using only a single forward and backward pass.…”

“…Recently techniques have been introduced that pool these individual pixel scores into more interpretable regions [10,13]. Of these, Superpixels Weighted by Average Gradient (SWAG) [10], is able to produce explanations efficiently using only a single forward and backward pass. We believe that SWAG can serve as a basis to create accurate and interpretable explanations for networks that take videos as an input.…”

Visually explaining 3D-CNN predictions for video classification with an adaptive occlusion sensitivity analysis

“…It has been shown that the separation of the information from Grad-CAM into spatial and temporal information works effectively for a more detailed explanation [13]. SWAG-V [12] enhances the framework SWAG [11] by averaging and smoothing a saliency map at the super-pixel level.…”

Convolutional Neural Network Interpretability with General Pattern Theory

“…The quality of heatmaps has been measured by how much the heatmaps improve ILSVRC localization (Zhou et al, 2016;Selvaraju et al, 2016). Metrics have been designed to measure changes after the manipulation of pixels with high relevance according to the XAI methods used (Bach et al, 2015;Samek et al, 2017;Hooker et al, 2019;Hartley et al, 2021), one of which, the most relevant first (MoRF) will be adopted here. We also use weight randomization sanity check in (Adebayo et al, 2018), where similarity is measured between explanations given before and after layer weight modification, for example, using rank correlation.…”