Visual Explanations from Deep Networks via Riemann-Stieltjes Integrated Gradient-based Localization

Abstract

Neural networks are becoming increasingly better at tasks that involve classifying and recognizing images. At the same time techniques intended to explain the network output have been proposed. One such technique is the Gradient-based Class Activation Map (Grad-CAM), which is able to locate features of an input image at various levels of a convolutional neural network (CNN), but is sensitive to the vanishing gradients problem. There are techniques such as Integrated Gradients (IG), that are not affected by that problem, but its use is limited to the input layer of a network. Here we introduce a new technique to produce visual explanations for the predictions of a CNN. Like Grad-CAM, our method can be applied to any layer of the network, and like Integrated Gradients it is not affected by the problem of vanishing gradients. For efficiency, gradient integration is performed numerically at the layer level using a Riemann-Stieltjes sum approximation. Compared to Grad-CAM, heatmaps produced by our algorithm are better focused in the areas of interest, and their numerical computation is more stable. Our code is available at https://github.com/mlerma54/RSIGradCAM.

Explainable Deep Learning Approach for Multi-Class Brain Magnetic Resonance Imaging Tumor Classification and Localization Using Gradient-Weighted Class Activation Mapping

“…However, these models lack explainability [21,22,31,46]. Although many XDL methods have been proposed for natural image problems [47][48][49], relatively less attention has been paid to model explainability in the context of brain imaging applications [19,50]. Consequently, the lack of interpretability in the models has been a concern for radiologists and healthcare professionals that find the black-box nature of the models inadequate for their needs.…”

Graph Adversarial Transfer Learning for Diabetic Retinopathy Classification

“…To further evaluate the performance of GATL, we visualized the Class Activation Map (CAM) of the features extracted by GATL, as shown in Figure 10. CAM is a tool that helps researchers [51] visualize CNNs. It can clearly show the image regions that the network is focusing on.…”