Explaining nonlinear classification decisions with deep Taylor decomposition

Montavon, Grégoire; Lapuschkin, Sebastian; Binder, Alexander; Samek, Wojciech; Müller, Klaus‐Robert

doi:10.1016/j.patcog.2016.11.008

Cited by 1,150 publications

(1,006 citation statements)

References 34 publications

Supporting

Mentioning

957

Contrasting

Unclassified

Order By: Relevance

“…Another future direction would be to analyze the interpretability of NNS systems, specifically for recommender systems with non-linear query mechanism, in terms of salient features that have led to the query result. This is in the line of research on ''explaining learning machines'', i.e., answering to the question which part of the data is responsible for specific decisions made by learning machines (Baehrens et al 2010;Zeiler and Fergus 2014;Bach et al 2015; Ribeiro et al 2016; Montavon et al 2017Montavon et al , 2018. This question is non-trivial when the learning machines are complex and non-linear.…”

Section: Resultsmentioning

confidence: 97%

Sharing hash codes for multiple purposes

Pronobis

Panknin

Kirschnick

et al. 2018

Jpn J Stat Data Sci

Self Cite

View full text Add to dashboard Cite

Locality sensitive hashing (LSH) is a powerful tool in data science, which enables sublinear-time approximate nearest neighbor search. A variety of hashing schemes have been proposed for different dissimilarity measures. However, hash codes significantly depend on the dissimilarity, which prohibits users from adjusting the dissimilarity at query time. In this paper, we propose multiple purpose LSH (mp-LSH) which shares the hash codes for different dissimilarities. mp-LSH supports L2, cosine, and inner product dissimilarities, and their corresponding weighted sums, where the weights can be adjusted at query time. It also allows us to modify the importance of pre-defined groups of features. Thus, mp-LSH enables us, for example, to retrieve similar items to a query with the user preference taken into account, to find a similar material to a query with some properties (stability, utility, etc.) optimized, and to turn on or off a part of multi-modal information (brightness, color, audio, text, etc.) in image/video retrieval. We theoretically and empirically analyze the performance of three variants of mp-LSH, and demonstrate their usefulness on real-world data sets.

show abstract

Section: Resultsmentioning

confidence: 97%

Sharing hash codes for multiple purposes

Pronobis

Panknin

Kirschnick

et al. 2018

Jpn J Stat Data Sci

Self Cite

View full text Add to dashboard Cite

show abstract

“…This redistribution rule has been showed to fulfill the layer-wise conservation property [10] and to be closely related to a deep variant of Taylor decomposition [11].…”

Section: B Interpretabilitymentioning

confidence: 93%

“…LRP explains individual classification decisions of a DNN by decomposing its output in terms of input variables. It is a principled method which has close relation to Taylor decomposition [11] and is applicable to arbitrary DNN architectures. From a practitioners perspective LRP adds a new dimension to the application of DNNs (e.g., in computer vision [12], [13]) by making the prediction transparent.…”

Section: Introductionmentioning

confidence: 99%

Interpretable deep neural networks for single-trial EEG classification

Sturm

Lapuschkin

Samek

et al. 2016

Journal of Neuroscience Methods

Self Cite

331

246

View full text Add to dashboard Cite

Abstract-Background: In cognitive neuroscience the potential of Deep Neural Networks (DNNs) for solving complex classification tasks is yet to be fully exploited. The most limiting factor is that DNNs as notorious 'black boxes' do not provide insight into neurophysiological phenomena underlying a decision. Layerwise Relevance Propagation (LRP) has been introduced as a novel method to explain individual network decisions. New Method: We propose the application of DNNs with LRP for the first time for EEG data analysis. Through LRP the singletrial DNN decisions are transformed into heatmaps indicating each data point's relevance for the outcome of the decision. Results: DNN achieves classification accuracies comparable to those of CSP-LDA. In subjects with low performance subjectto-subject transfer of trained DNNs can improve the results. The single-trial LRP heatmaps reveal neurophysiologically plausible patterns, resembling CSP-derived scalp maps. Critically, while CSP patterns represent class-wise aggregated information, LRP heatmaps pinpoint neural patterns to single time points in single trials. Comparison with Existing Method(s):We compare the classification performance of DNNs to that of linear CSP-LDA on two data sets related to motor-imaginery BCI. Conclusion: We have demonstrated that DNN is a powerful nonlinear tool for EEG analysis. With LRP a new quality of highresolution assessment of neural activity can be reached. LRP is a potential remedy for the lack of interpretability of DNNs that has limited their utility in neuroscientific applications. The extreme specificity of the LRP-derived heatmaps opens up new avenues for investigating neural activity underlying complex perception or decision-related processes.

show abstract

“…It is a principled method that has a close relationship to Taylor decomposition and is applicable to arbitrary deep neural network architectures [30]. The output is a heatmap over the input features that indicates the relevance of each feature to the model output.…”

Section: B Model Functionalitymentioning

confidence: 99%

Interpretability of deep learning models: A survey of results

Chakraborty

Tomsett

Raghavendra

et al. 2017

2017 IEEE SmartWorld, Ubiquitous Intelligence &Amp; Computing, Advanced &Amp; Trusted Computed, Scalable Computing &Amp; Commun

278

189

View full text Add to dashboard Cite

Abstract-Deep neural networks have achieved near-human accuracy levels in various types of classification and prediction tasks including images, text, speech, and video data. However, the networks continue to be treated mostly as black-box function approximators, mapping a given input to a classification output. The next step in this human-machine evolutionary processincorporating these networks into mission critical processes such as medical diagnosis, planning and control -requires a level of trust association with the machine output.Typically, statistical metrics are used to quantify the uncertainty of an output. However, the notion of trust also depends on the visibility that a human has into the working of the machine. In other words, the neural network should provide humanunderstandable justifications for its output leading to insights about the inner workings. We call such models as interpretable deep networks.Interpretability is not a monolithic notion. In fact, the subjectivity of an interpretation, due to different levels of human understanding, implies that there must be a multitude of dimensions that together constitute interpretability. In addition, the interpretation itself can be provided either in terms of the lowlevel network parameters, or in terms of input features used by the model. In this paper, we outline some of the dimensions that are useful for model interpretability, and categorize prior work along those dimensions. In the process, we perform a gap analysis of what needs to be done to improve model interpretability.

show abstract

Explaining nonlinear classification decisions with deep Taylor decomposition

Cited by 1,150 publications

References 34 publications

Sharing hash codes for multiple purposes

Sharing hash codes for multiple purposes

Interpretable deep neural networks for single-trial EEG classification

Interpretability of deep learning models: A survey of results

Contact Info

Product

Resources

About