Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations

Abstract

The convolutional neural network (CNN) has become a powerful tool for various biomedical image analysis tasks, but there is a lack of visual explanation for the machinery of CNNs. In this paper, we present a novel algorithm, Respond-weighted Class Activation Mapping (Respond-CAM), for making CNN-based models interpretable by visualizing input regions that are important for predictions, especially for biomedical 3D imaging data inputs. Our method uses the gradients of any target concept (e.g. the score of target class) that flows into a convolutional layer. The weighted feature maps are combined to produce a heatmap that highlights the important regions in the image for predicting the target concept. We prove a preferable sum-to-score property of the Respond-CAM and verify its significant improvement on 3D images from the current state-of-the-art approach. Our tests on Cellular Electron Cryo-Tomography 3D images show that Respond-CAM achieves superior performance on visualizing the CNNs with 3D biomedical images inputs, and is able to get reasonably good results on visualizing the CNNs with natural image inputs. The Respond-CAM is an efficient and reliable approach for visualizing the CNN machinery, and is applicable to a wide variety of CNN model families and image analysis tasks. Our code is available at:

A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI

“…In general, they decompose signals propagated within their algorithm and selectively rearrange and process them to provide interpretable information. Class activation map (CAM) has been a popular method to generate heat/saliency/relevance-map (from now, we will use the terms interchangeably) that corresponds to discriminative features for classifications [42]- [44]. The original implementation of CAM [42] produces heatmaps using f k (x, y), the pixel-wise activation of unit k across spatial coordinates (x, y) in the last convolutional layers, weighted by w c k , the coefficient corresponding to unit k for class c. CAM at pixel (x, y) is thus given by M c (x, y) = k w c k f k (x, y).…”

“…For NLP or sentiment analysis, saliency map can also take the form of "heat" scores over words in texts, as demonstrated by Arras et al [62] using LRP and by Karpathy et al [63]. In the medical field (see later section), Irvin et al [6], Zhao et al [44], Paschali et al [64], Couture et al [65], Li et al [66], Qin et al [67], Tang et al [68], Papanastasopoulos et al [69], and Lee et al [70] have studied methods employing saliency and visual explanations. It is noted that we also subcategorize LIME as a method that uses optimization and sensitivity as its underlying mechanisms, and many researches on interpretability span more than one subcategories.…”

“…c) Functions-based: Third, an evaluation is functionally grounded if there exist proxies (which can be defined a priori) for evaluation, for example, sparsity [10]. Some articles [2], [5], [42]- [44], [96], [97], [144], and [145] use metrics that rely on this evaluation include many supervised learning models with clearly defined metrics such as: 1) dice coefficients (related to visual interpretability) and 2) attribution values, components of canonically transformed variables (see for example CCA) or values obtained from dimensionality reduction methods (such as components of principal components from PCA and their corresponding eigenvalues), where interpretability is related to the degree an object relates to a feature, for example, classification of a dog has high values in the feature space related to four limbs, shape of snout and paws, etc. Which suitable metrics to use are highly dependent on the tasks at hand.…”

“…Many more works have thus been dedicated to exploring explainability in the medical fields [11], [20], [44]. They provide summaries of previous works [21] including subfield-specific reviews such as [25] for chest radiograph and sentiment analysis in medicine [161], or at least set aside a section to promote awareness for the importance of interpretability in the medical field [162].…”

Going Deep in Medical Image Analysis: Concepts, Methods, Challenges, and Future Directions

“…The developed AIR network is also evaluated on clinical datasets acquired through image-fusion guided prostate biopsy procedures. For the visualization of 3D medical image data Zhao et al [216] recently proposed a deep learning based technique, named Respond-weighted Class Activation Mapping (Respond-CAM). As compared to Grade-CAM [217] they claim better performance.…”