Towards Better Explanations of Class Activation Mapping

Jung, Hun Soon; Oh, Youngrock

doi:10.1109/iccv48922.2021.00137

Cited by 60 publications

(42 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These metrics consist to multiply the input image with an explanation map to mask the non-relevant areas and to measure the class score variation. Jung et al proposed a variant of AD where the salient areas are masked instead of the nonsalient, called Average Drop in Deletion (ADD) [15]. In parallel, Petsiuk et al proposed DAUC and IAUC which study the score variation while progressively masking/revealing the image instead of applying the saliency map once [22].…”

Section: Existing Metricsmentioning

confidence: 99%

“…Various metrics have been proposed to automatically evaluate saliency maps generated by explanation methods [4,15,22]. These metrics consist to add or remove the important areas according to the saliency map and measure the impact on the initially predicted class score.…”

Section: Existing Metricsmentioning

confidence: 99%

“…Recent years have seen a surge of interest in interpretable machine learning, as most state-of-the-art learning models currently are deep models [3,17,24,25] and suffer from their lack of interpretability due to their black-box nature. In image classification, many generic approaches have been proposed to explain a model's decision by generating saliency maps that highlight the important areas of the image with regards to the task at hand [1,4,15,18,22,23,27,31]. The community of interpretable deep learning has yet to find a consensus about Supported by Nantes Excellence Trajectory (NExT) how to evaluate these methods, the main difficulty residing in the ambiguity of the concept of interpretability.…”

Section: Introductionmentioning

confidence: 99%

“…Because of these issues, another trend proposes to design objective metrics to evaluate generic explanation methods [4,15,22]. In this paper, we chose to follow this trend, by proposing three new metrics.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Metrics for saliency map evaluation of deep learning explanation methods

Gomez¹,

Fréour²,

Mouchère³

2022

Preprint

View full text Add to dashboard Cite

Due to the black-box nature of deep learning models, there is a recent development of solutions for visual explanations of CNNs. Given the high cost of user studies, metrics are necessary to compare and evaluate these different methods. In this paper, we critically analyze the Deletion Area Under Curve (DAUC) and Insertion Area Under Curve (IAUC) metrics proposed by Petsiuk et al. (2018). These metrics were designed to evaluate the faithfulness of saliency maps generated by generic methods such as Grad-CAM or RISE. First, we show that the actual saliency score values given by the saliency map are ignored as only the ranking of the scores is taken into account. This shows that these metrics are insufficient by themselves, as the visual appearance of a saliency map can change significantly without the ranking of the scores being modified. Secondly, we argue that during the computation of DAUC and IAUC, the model is presented with images that are out of the training distribution which might lead to an unreliable behavior of the model being explained. To complement DAUC/IAUC, we propose new metrics that quantify the sparsity and the calibration of explanation methods, two previously unstudied properties. Finally, we give general remarks about the metrics studied in this paper and discuss how to evaluate them in a user study.

show abstract

Section: Existing Metricsmentioning

confidence: 99%

Section: Existing Metricsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Metrics for saliency map evaluation of deep learning explanation methods

Gomez¹,

Fréour²,

Mouchère³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…From the viewpoint of assigning attributions [8], [21], [22], [23], [24], [1] first provided the concept of relevance and conservation for the evidence to the decision using several types of layer-wise relevance propagation (LRP) rules. [3] proposed deep Taylor decomposition (DTD) based on Taylor expansion in intermediate layers with a theoretical foundation.…”

Section: Related Studiesmentioning

confidence: 99%

Gradient Hedging for Intensively Exploring Salient Interpretation beyond Neuron Activation

Nam¹,

Lee²

2022

Preprint

View full text Add to dashboard Cite

Hedging is a strategy for reducing the potential risks in various types of investments by adopting an opposite position in a related asset. Motivated by the equity technique, we introduce a method for decomposing output predictions into intensive salient attributions by hedging the evidence for a decision. We analyze the conventional approach applied to the evidence for a decision and discuss the paradox of the conservation rule. Subsequently, we define the viewpoint of evidence as a gap of positive and negative influence among the gradient-derived initial contribution maps and propagate the antagonistic elements to the evidence as suppressors, following the criterion of the degree of positive attribution defined by user preference. In addition, we reflect the severance or sparseness contribution of inactivated neurons, which are mostly irrelevant to a decision, resulting in increased robustness to interpretability. We conduct the following assessments in a verified experimental environment: pointing game, most relevant first region insertion, outside-inside relevance ratio, and mean average precision on the PASCAL VOC 2007, MS COCO 2014, and ImageNet datasets. The results demonstrate that our method outperforms existing attribution methods in distinctive, intensive, and intuitive visualization with robustness and applicability in general models.

show abstract

Assessing retinal vein occlusion based on color fundus photographs using neural understanding network (NUN)

et al. 2022

View full text Add to dashboard Cite

Objective To develop and validate a novel deep learning architecture to classify retinal vein occlusion (RVO) on color fundus photographs (CFPs) and reveal the image features contributing to the classification. Methods The neural understanding network (NUN) is formed by two components: (1) convolutional neural network (CNN)‐based feature extraction and (2) graph neural networks (GNN)‐based feature understanding. The CNN‐based image features were transformed into a graph representation to encode and visualize long‐range feature interactions to identify the image regions that significantly contributed to the classification decision. A total of 7062 CFPs were classified into three categories: (1) no vein occlusion (“normal”), (2) central RVO, and (3) branch RVO. The area under the receiver operative characteristic (ROC) curve (AUC) was used as the metric to assess the performance of the trained classification models. Results The AUC, accuracy, sensitivity, and specificity for NUN to classify CFPs as normal, central occlusion, or branch occlusion were 0.975 (± 0.003), 0.911 (± 0.007), 0.983 (± 0.010), and 0.803 (± 0.005), respectively, which outperformed available classical CNN models. Conclusion The NUN architecture can provide a better classification performance and a straightforward visualization of the results compared to CNNs.

show abstract

Towards Better Explanations of Class Activation Mapping

Cited by 60 publications

References 13 publications

Metrics for saliency map evaluation of deep learning explanation methods

Metrics for saliency map evaluation of deep learning explanation methods

Gradient Hedging for Intensively Exploring Salient Interpretation beyond Neuron Activation

Assessing retinal vein occlusion based on color fundus photographs using neural understanding network (NUN)

Contact Info

Product

Resources

About