Saliency-driven Class Impressions for Feature Visualization of Deep Neural Networks

Abstract

In this paper, we propose a data-free method of extracting Impressions of each class from the classifier's memory. The Deep Learning regime empowers classifiers to extract distinct patterns (or features) of a given class from training data, which is the basis on which they generalize to unseen data. Before deploying these models on critical applications, it is very useful to visualize the features considered to be important for classification. Existing visualization methods develop high confidence images consisting of both background and foreground features. This makes it hard to judge what the important features of a given class are. In this work, we propose a saliency-driven approach to visualize discriminative features that are considered most important for a given task. Another drawback of existing methods is that, confidence of the generated visualizations is increased by creating multiple instances of the given class. We restrict the algorithm to develop a single object per image, which helps further in extracting features of high confidence, and also results in better visualizations. We further demonstrate the generation of negative images as naturally fused images of two or more classes.

Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization

“…Here, we mention a few selected methods as examples of the plethora of approaches for understanding CNN decision-making: Saliency maps show the importance of each pixel to the classification decision (Springenberg et al, 2014;Bach et al, 2015;Smilkov et al, 2017;Zintgraf et al, 2017), concept activation vectors show a model's sensitivity to human-defined concepts (Kim et al, 2018), and other methods -amongst feature visualizations -focus on explaining individual units Bau et al (2020). Some tools integrate interactive, software-like aspects (Hohman et al, 2019;Wang et al, 2020;Carter et al, 2019;Collaris & van Wijk, 2020;OpenAI, 2020), combine more than one explanation method (Shi et al, 2020;Addepalli et al, 2020) or make progress towards automated explanation methods (Lapuschkin et al, 2019;Ghorbani et al, 2019b). As overviews, we recommend Zhang & Zhu (2018); Montavon et al (2018) and Samek et al (2020).…”

“…As such, a particular milestone for CNNs was understanding that features are formed in a hierarchical fashion (LeCun et al, 2015;Güc ¸lü & van Gerven, 2015;Goodfellow et al, 2016). Over the past few years, extensive investigations to better understand CNNs are based on feature visualizations (Olah et al, 2020b;a;Cammarata et al, 2020;Cadena et al, 2018), and the technique is being combined with other explanation methods (Olah et al, 2018;Carter et al, 2019;Addepalli et al, 2020;Hohman et al, 2019).…”