Building Interpretable Interaction Trees for Deep NLP Models

Abstract

This paper proposes a method to disentangle and quantify interactions among words that are encoded inside a DNN for natural language processing. We construct a tree to encode salient interactions extracted by the DNN. Six metrics are proposed to analyze properties of interactions between constituents in a sentence. The interaction is defined based on Shapley values of words, which are considered as an unbiased estimation of word contributions to the network prediction. Our method is used to quantify word interactions encoded inside the BERT, ELMo, LSTM, CNN, and Transformer networks. Experimental results have provided a new perspective to understand these DNNs, and have demonstrated the effectiveness of our method.

Visual Explanation for Open-domain Question Answering with BERT

“…Additionally, using hierarchical representations to analyze DNN has become an area of interest in the NLP community. Zhang et al [93] constructed a tree to encode salient interactions extracted by DNN, on the basis of Shapley values of words [50]. Considering that displaying the original attribution matrix in the case of long text causes visual clutter and that most elements of the attribution matrix are minimal (close to zero), we adopt the tree generation algorithm proposed by Hao et al [25] to display the information flow inside the module (R4).…”

Explaining Interactions Between Text Spans

“…Sikdar et al (2021) compute importance scores in a bottom-up manner starting from the individual embedding dimensions, working its way up to tokens, words, phrases, and finally the sentence. Zhang et al (2021) build interpretable interaction trees, where the interaction is again defined based on Shapley values. While these methods produce spans of tokens that are part of an interaction, the hierarchical nature of the explanation limits the interactions only to neighboring spans.…”