Understanding the role of individual units in a deep neural network

Abstract

Deep neural networks excel at finding hierarchical representations that solve complex tasks over large datasets. How can we humans understand these learned representations? In this work, we present network dissection, an analytic framework to systematically identify the semantics of individual hidden units within image classification and image generation networks. First, we analyze a convolutional neural network (CNN) trained on scene classification and discover units that match a diverse set of object concepts. We find evidence that the network has learned many object classes that play crucial roles in classifying scene classes. Second, we use a similar analytic method to analyze a generative adversarial network (GAN) model trained to generate scenes. By analyzing changes made when small sets of units are activated or deactivated, we find that objects can be added and removed from the output scenes while adapting to the context. Finally, we apply our analytic framework to understanding adversarial attacks and to semantic image editing.

Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications

“…Further methods do not seek to explain in terms of input features but in terms of the latent space, where the directions in the latent space code for higher level concepts, such as color, material, object part, or object [17], [18], [205]. In particular, the TCAV method [89] produces a latent-space explanation for every individual prediction.…”

“…Let us use for this the Adience benchmark data set [44], providing 26 580 images captured "in the wild" and labeled into eight ordinal groups of age ranges {(0-2), (4-6), (8)(9)(10)(11)(12)(13), (15)(16)(17)(18)(19)(20), (25)(26)(27)(28)(29)(30)(31)(32), (38)(39)(40)(41)(42)(43), (48)(49)(50)(51)(52)(53), (60+)}.…”

“…Orthogonal to the quest for better and more holistic ML models, explainable artificial intelligence (XAI) [18], [73], [112], [128], [159] has developed as a subfield of ML that seeks to augment the training process, the learned representations, and the decisions with human-interpretable explanations. An example is medical diagnosis, where the input examples (e.g., histopathological images) come with various artifacts (e.g., stemming from image quality or suboptimal annotations) that have, in principle, nothing to do with the diagnostic task, yet, due to the limited amount of available data, the ML model may harvest specifically these spurious correlations with the prediction target (e.g., [61] and [177]).…”

A Survey on Applications of Reinforcement Learning in Flying Ad-Hoc Networks

“…Every neuron has a weighted input, an activation function, and an output. The activation function determines the output depending on the input of the neuron [22]. It acts as a trigger that depends on the weighted input.…”

The Role of Edges in Line Drawing Perception

“…Hence, a second prediction is that an accurate neural model should have similar cortical responses to a line drawing as to they do to a corresponding realistic image. Moreover, probing how these networks operate, for example, as in Zeiler and Fergus (2014) and Bau et al. (2020), could provide more precise understanding of how precisely line drawings are interpreted.…”