A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks

Shahroudnejad, Atefeh

doi:10.48550/arxiv.2102.01792

Cited by 6 publications

(6 citation statements)

References 44 publications

(59 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several works have been done in the area of feature extraction, especially applied for model interpretability and explainability (Gilpin et al, 2019 ; Fan et al, 2021 ; Ismail et al, 2021 ; Shahroudnejad, 2021 ; Thakur and Han, 2021 ). These techniques, used for extracting features, usually consist in activation maximization (Mahendran and Vedaldi, 2016 ; Ellis et al, 2021 ), where a group of neurons, which can involve from a single neuron up to an entire layer (or channel for convolutions), is selected to extract the feature by maximizing its activation.…”

Section: Related Workmentioning

confidence: 99%

Feature alignment as a generative process

Farias

Maziero²

2023

Front. Artif. Intell.

View full text Add to dashboard Cite

Reversibility in artificial neural networks allows us to retrieve the input given an output. We present feature alignment, a method for approximating reversibility in arbitrary neural networks. We train a network by minimizing the distance between the output of a data point and the random output with respect to a random input. We applied the technique to the MNIST, CIFAR-10, CelebA, and STL-10 image datasets. We demonstrate that this method can roughly recover images from just their latent representation without the need of a decoder. By utilizing the formulation of variational autoencoders, we demonstrate that it is possible to produce new images that are statistically comparable to the training data. Furthermore, we demonstrate that the quality of the images can be improved by coupling a generator and a discriminator together. In addition, we show how this method, with a few minor modifications, can be used to train networks locally, which has the potential to save computational memory resources.

show abstract

Section: Related Workmentioning

confidence: 99%

Feature alignment as a generative process

Farias

Maziero²

2023

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…Several works have been done in the area of feature extraction, especially applied for model interpretability and explainability [12][13][14] . These techniques, used for extracting features, usually consist in activation maximization 15 , where a group of neurons, which can involve from a single neuron up to an entire layer (or channel for convolutions), is selected to extract the feature by maximizing its activation.…”

Section: Related Workmentioning

confidence: 99%

Feature Alignment As A Generative Process

Farias

Maziero

2022

Preprint

View full text Add to dashboard Cite

We introduce feature alignment, a technique for obtaining approximate reversibility in artificial neural networks. By means of feature extraction, we can train a neural network to learn an estimated map for its reverse process from outputs to inputs. Combined with variational autoencoders, we can generate new samples from the same statistics as the training data. Improvements of the results are obtained by using concepts from generative adversarial networks. Finally, we show that the technique can be modified for training neural networks locally, saving computational memory resources. Applying these techniques, we report results for three vision generative tasks: MNIST, CIFAR-10, and celebA.

show abstract

“…Many prior studies [40], [41], [42], [43], [44], [45], [46], [14], [15] have been proposed to analyze and explain the behaviors of deep neural network. Functional analysis and decision analysis are two main categories of analysis of DNN [47]. Functional analysis, i.e., black-box analysis, aims to capture the overall behavior by investigating the relation between inputs and outputs [41], [43], [48].…”

Section: Threats To Validitymentioning

confidence: 99%

DeepMemory: Model-based Memorization Analysis of Deep Neural Language Models

Zhu

Chen

Shang

et al. 2021

2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

The neural network model is having a significant impact on many real-world applications. Unfortunately, the increasing popularity and complexity of these models also amplifies their security and privacy challenges, with privacy leakage from training data being one of the most prominent issues. In this context, prior studies proposed to analyze the abstraction behavior of neural network models, e.g., RNN, to understand their robustness. However, the existing research rarely addresses privacy breaches caused by memorization in neural language models. To fill this gap, we propose a novel approach, DeepMemory, that analyzes memorization behavior for a neural language model. We first construct a memorization-analysis-oriented model, taking both training data and a neural language model as input. We then build a semantic first-order Markov model to bind the constructed memorization-analysis-oriented model to the training data to analyze memorization distribution. Finally, we apply our approach to address data leakage issues associated with memorization and to assist in dememorization. We evaluate our approach on one of the most popular neural language models, the LSTM-based language model, with three public datasets, namely, WikiText-103, WMT2017, and IWSLT2016. We find that sentences in the studied datasets with low perplexity are more likely to be memorized. Our approach achieves an average AUC of 0.73 in automatically identifying data leakage issues during assessment. We also show that with the assistance of DeepMemory, data breaches due to memorization of neural language models can be successfully mitigated by mutating training data without reducing the performance of neural language models.

show abstract

A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks

Cited by 6 publications

References 44 publications

Feature alignment as a generative process

Feature alignment as a generative process

Feature Alignment As A Generative Process

DeepMemory: Model-based Memorization Analysis of Deep Neural Language Models

Contact Info

Product

Resources

About