LayerCAM: Exploring Hierarchical Class Activation Maps for Localization

Jiang, Peng-Tao; Zhang, Changbin; Hou, Qibin; Cheng, Ming–Ming; Wei, Yunchao

doi:10.1109/tip.2021.3089943

Cited by 378 publications

(211 citation statements)

References 84 publications

Supporting

Mentioning

165

Contrasting

Unclassified

Order By: Relevance

“…Explaining the classification prediction. We leverage three different methods (i.e., Layer-CAM [54], integrated gradient [55] and extremal perturbation [56]) to explain the image classification prediction obtained by ResNet-50 trained on the Good & Bad bird data set and ResNet-101 trained on the Good & Bad face data set. Fig.…”

Section: Results On the Good And Bad Data Setmentioning

confidence: 99%

OptGAN: Optimizing and Interpreting the Latent Space of the Conditional Text-to-Image GANs

Zhang¹,

Schomaker²

2022

Preprint

View full text Add to dashboard Cite

Text-to-image generation intends to automatically produce a photo-realistic image, conditioned on a textual description. It can be potentially employed in the field of art creation, data augmentation, photo-editing, etc. Although many efforts have been dedicated to this task, it remains particularly challenging to generate believable, natural scenes. To facilitate the real-world applications of text-toimage synthesis, we focus on studying the following three issues: 1) How to ensure that generated samples are believable, realistic or natural? 2) How to exploit the latent space of the generator to edit a synthesized image? 3) How to improve the explainability of a text-to-image generation framework? In this work, we constructed two novel data sets (i.e., the Good & Bad bird and face data sets) consisting of successful as well as unsuccessful generated samples, according to strict criteria. To effectively and efficiently acquire high-quality images by increasing the probability of generating latent codes, we use a dedicated Good/Bad classifier for generated images. It is based on a pre-trained front end and fine-tuned on the basis of the proposed Good & Bad data set. After that, we present a novel algorithm which identifies semantically-understandable directions in the latent space of a conditional text-to-image GAN architecture by performing independent component analysis on the pre-trained weight values of the generator. Furthermore, we develop a backgroundflattening loss (BFL), to improve the background appearance in the edited image. Subsequently, we introduce linear interpolation analysis between pairs of keywords. This is extended into a similar triangular 'linguistic' interpolation in order to take a deep look into what a text-to-image synthesis model has learned within the linguistic embeddings. Experimental results on the recent Diver-GAN generator pre-trained on three benchmark data sets demonstrate that our classifier achieves a better than 98% accuracy in predicting Good/Bad classes for synthetic samples and our proposed approach is able to derive various interpretable semantic properties for a conditional text-to-image GAN model, confirming the effectiveness of our presented techniques. Our data set is available at https://zenodo.org/record/6283798#.YhkN_ujMI2w.

show abstract

Section: Results On the Good And Bad Data Setmentioning

confidence: 99%

OptGAN: Optimizing and Interpreting the Latent Space of the Conditional Text-to-Image GANs

Zhang¹,

Schomaker²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Malolan et al [44] showed also that LRP does not highlight recognizable structures in the image for DNNs based on the Xception architecture. The methods Grad-CAM [32] and LayerCAM [48] can only mark coarse regions as relevant for the decision-making process. Thus, these methods are not suitable for localizing the exact structures corresponding to traces of forgery.…”

Section: Sample Resultsmentioning

confidence: 99%

“…Thus, these methods are not suitable for localizing the exact structures corresponding to traces of forgery. input image LRP Xception [43] Grad-CAM [32] LayerCam [48] Figure 8. LRP relevance assignment results for the Xception architecture DNN and Grad-CAM and LayerCAM for the Naïve Detector applied on the same images shown in Figures 6 and 7.…”

Section: Sample Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Feature Focus: Towards Explainable and Transparent Deep Face Morphing Attack Detectors

2021

View full text Add to dashboard Cite

Detecting morphed face images has become an important task to maintain the trust in automated verification systems based on facial images, e.g., at automated border control gates. Deep Neural Network (DNN)-based detectors have shown remarkable results, but without further investigations their decision-making process is not transparent. In contrast to approaches based on hand-crafted features, DNNs have to be analyzed in complex experiments to know which characteristics or structures are generally used to distinguish between morphed and genuine face images or considered for an individual morphed face image. In this paper, we present Feature Focus, a new transparent face morphing detector based on a modified VGG-A architecture and an additional feature shaping loss function, as well as Focused Layer-wise Relevance Propagation (FLRP), an extension of LRP. FLRP in combination with the Feature Focus detector forms a reliable and accurate explainability component. We study the advantages of the new detector compared to other DNN-based approaches and evaluate LRP and FLRP regarding their suitability for highlighting traces of image manipulation from face morphing. To this end, we use partial morphs which contain morphing artifacts in predefined areas only and analyze how much of the overall relevance each method assigns to these areas.

show abstract

“…Grad-CAM relies on up-sampling the final class activation map and hence suffers from poor resolution. Recently, LayerCAM has been proposed as an effective means to mitigate this limitation by weighting each image pixel using the backward class-specific gradients, yielding more fine-grained visualizations (Jiang et al, 2021). However, saliency map reproducibility and localization ability has previously been criticized in the context of complex models trained on medical image classification tasks (Arun et al, 2021).…”

Section: Discussionmentioning

confidence: 99%

Interpretability Aware Model Training to Improve Robustness against Out-of-Distribution Magnetic Resonance Images in Alzheimer's Disease Classification

Kuijs¹,

Jutzeler²,

Rieck³

et al. 2021

Preprint

View full text Add to dashboard Cite

Owing to its pristine soft-tissue contrast and high resolution, structural magnetic resonance imaging (MRI) is widely applied in neurology, making it a valuable data source for imagebased machine learning (ML) and deep learning applications. The physical nature of MRI acquisition and reconstruction, however, causes variations in image intensity, resolution, and signal-to-noise ratio. Since ML models are sensitive to such variations, performance on out-of-distribution data, which is inherent to the setting of a deployed healthcare ML application, typically drops below acceptable levels. We propose an interpretability aware adversarial training regime to improve robustness against out-of-distribution samples originating from different MRI hardware. The approach is applied to 1.5T and 3T MRIs obtained from the Alzheimer's Disease Neuroimaging Initiative database. We present preliminary results showing promising performance on out-ofdistribution samples.

show abstract

LayerCAM: Exploring Hierarchical Class Activation Maps for Localization

Cited by 378 publications

References 84 publications

OptGAN: Optimizing and Interpreting the Latent Space of the Conditional Text-to-Image GANs

OptGAN: Optimizing and Interpreting the Latent Space of the Conditional Text-to-Image GANs

Feature Focus: Towards Explainable and Transparent Deep Face Morphing Attack Detectors

Interpretability Aware Model Training to Improve Robustness against Out-of-Distribution Magnetic Resonance Images in Alzheimer's Disease Classification

Contact Info

Product

Resources

About