The heads hypothesis: A unifying statistical approach towards understanding multi-headed attention in BERT

Abstract

Multi-headed attention heads are a mainstay in transformerbased models. Different methods have been proposed to classify the role of each attention head based on the relations between tokens which have high pair-wise attention. These roles include syntactic (tokens with some syntactic relation), local (nearby tokens), block (tokens in the same sentence) and delimiter (the special [CLS], [SEP] tokens). There are two main challenges with existing methods for classification: (a) there are no standard scores across studies or across functional roles, and (b) these scores are often average quantities measured across sentences without capturing statistical significance. In this work, we formalize a simple yet effective score that generalizes to all the roles of attention heads and employs hypothesis testing on this score for robust inference. This provides us the right lens to systematically analyze attention heads and confidently comment on many commonly posed questions on analyzing the BERT model. In particular, we comment on the co-location of multiple functional roles in the same attention head, the distribution of attention heads across layers, and effect of fine-tuning for specific NLP tasks on these functional roles. Code is made publicly available. 1

Input-specific Attention Subnetworks for Adversarial Detection

“…Which heads should we flip? We take motivation from studies that show that middle layers of BERT capture syntactic relations (Hewitt and Manning, 2019;Goldberg, 2019) and are multi-skilled (Pande et al, 2021), making them crucial for prediction. In contrast, the initial layers are responsible for phraselevel understanding while the last few layers are highly task-specific (Jawahar et al, 2019).…”

“…When comparing across datasets, we observe that AdvNet performs better on simpler sentence labelling datasets like SST-2 and AG News when compared to more complex tasks like RTE and MRPC which require comparison between sentences. Existing work (Pande et al, 2021) shows that for simpler tasks, the BERT heads perform discrete non-overlapping roles, while for complex tasks, there is greater overlap in head roles and a few heads perform more than one role. We hypothesize that this nature implies that the attention masks for different inputs even belonging to the same type (authentic or adversarial) can vary widely.…”

“…One, for a token in an input sentence, we can visualize the attention paid by a head to all other tokens. Such attention patterns are attractive linguistically and have come to define roles for attention heads (Pande et al, 2021). Two, the output of attention heads from various layers can be probed for their ability to encode information related to the "NLP pipeline" (Jawahar et al, 2019;Tenney et al, 2019;van Aken et al, 2019).…”