Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement

Poerner, Nina; Roth, Benjamin; Schütze, Hinrich

doi:10.18653/v1/p18-1032

Cited by 75 publications

(75 citation statements)

References 22 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative default strategy in computer vision is to uniformly employ the LRP-α 1 β 0 in every hidden layer [53], the latter has the advantage of having no free parameter, and delivers positive explanations. On convolutional neural networks for text, LRP-with a small value was found to work well [3,57], it provides a signed explanation.…”

Section: Lrp In Deep Neuralmentioning

confidence: 85%

“…For removing a word we simply discard it from the input sequence and concatenate the remaining parts of the sentence. An alternative removal scheme would have been to set the corresponding word embedding to zero in the input (which in practice gave us similar results), however the former enables us to generate more natural texts, although we acknowledge that the resulting sentence might be partly syntactically broken as pointed out by Poerner et al [57].…”

Section: Validating Explanations On Standard Lstms: Selectivity and Fmentioning

confidence: 99%

“…For another evaluation of recurrent neural networks explanations, including a standard LSTM model, we further refer to Poerner et al [57], in particular to their experiment using a subject-verb agreement task. Here the authors find that LRP and DeepLIFT [67] perform best among the tested explanation methods, both when using the signal-take-all strategy proposed in Arras et al [4] for product layers 5 .…”

Section: Validating Explanations On Standard Lstms: Selectivity and Fmentioning

confidence: 99%

See 2 more Smart Citations

Explaining and Interpreting LSTMs

Arras

Arjona-Medina

Widrich

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

While neural networks have acted as a strong unifying force in the design of modern AI systems, the neural network architectures themselves remain highly heterogeneous due to the variety of tasks to be solved. In this chapter, we explore how to adapt the Layer-wise Relevance Propagation (LRP) technique used for explaining the predictions of feed-forward networks to the LSTM architecture used for sequential data modeling and forecasting. The special accumulators and gated interactions present in the LSTM require both a new propagation scheme and an extension of the underlying theoretical framework to deliver faithful explanations.Keywords: Explainable Artificial Intelligence · Model Transparency · Recurrent Neural Networks · LSTM · Interpretability L. Arras and J. Arjona-Medina contributed equally to this work.The final authenticated publication is available online at https://doi.

show abstract

Section: Lrp In Deep Neuralmentioning

confidence: 85%

Section: Validating Explanations On Standard Lstms: Selectivity and Fmentioning

confidence: 99%

Section: Validating Explanations On Standard Lstms: Selectivity and Fmentioning

confidence: 99%

See 1 more Smart Citation

Explaining and Interpreting LSTMs

Arras

Arjona-Medina

Widrich

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…This section presents our automatic evaluation approach, which is an extension of the hybrid document paradigm (Poerner et al, 2018). The major advantage of automatic evaluation in the context of explanation methods is that it does not require manual annotation.…”

Section: Automatic Evaluation Using Fake Factsmentioning

confidence: 99%

“…The major advantage of automatic evaluation in the context of explanation methods is that it does not require manual annotation. Poerner et al (2018) create hybrid documents by randomly concatenating fragments of different documents. We adapt this paradigm to our use case in the following way:…”

Section: Automatic Evaluation Using Fake Factsmentioning

confidence: 99%

Interpretable Question Answering on Knowledge Bases and Text

Sydorova¹,

Poerner²,

Roth³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Interpretability of machine learning (ML) models becomes more relevant with their increasing adoption. In this work, we address the interpretability of ML based question answering (QA) models on a combination of knowledge bases (KB) and text documents. We adapt post hoc explanation methods such as LIME and input perturbation (IP) and compare them with the self-explanatory attention mechanism of the model. For this purpose, we propose an automatic evaluation paradigm for explanation methods in the context of QA. We also conduct a study with human annotators to evaluate whether explanations help them identify better QA models. Our results suggest that IP provides better explanations than LIME or attention, according to both automatic and human evaluation. We obtain the same ranking of methods in both experiments, which supports the validity of our automatic evaluation paradigm.

show abstract

Towards Explainable Artificial Intelligence

Samek

Müller

2019

Lecture Notes in Computer Science

548

357

View full text Add to dashboard Cite

In recent years, machine learning (ML) has become a key enabling technology for the sciences and industry. Especially through improvements in methodology, the availability of large databases and increased computational power, today's ML algorithms are able to achieve excellent performance (at times even exceeding the human level) on an increasing number of complex tasks. Deep learning models are at the forefront of this development. However, due to their nested non-linear structure, these powerful models have been generally considered "black boxes", not providing any information about what exactly makes them arrive at their predictions. Since in many applications, e.g., in the medical domain, such lack of transparency may be not acceptable, the development of methods for visualizing, explaining and interpreting deep learning models has recently attracted increasing attention. This introductory paper presents recent developments and applications in this field and makes a plea for a wider use of explainable learning algorithms in practice.

show abstract

Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement

Cited by 75 publications

References 22 publications

Explaining and Interpreting LSTMs

Explaining and Interpreting LSTMs

Interpretable Question Answering on Knowledge Bases and Text

Towards Explainable Artificial Intelligence

Contact Info

Product

Resources

About