Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?

Eberle, Oliver; Brandl, Stephanie; Pilot, Jonas; Søgaard, Anders

doi:10.18653/v1/2022.acl-long.296

Cited by 7 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…task difficulty and enjoyment) is an open question. Alternatively to actively annotating rationales, techniques like eye-tracking might allow for passive rationale collection (Eberle et al, 2022); for example, construct a heatmap of relevant text snippets based on the annotator's gaze while performing a task.…”

Section: E Ect On Annotation Timementioning

confidence: 99%

Human-annotated rationales and explainable text classification: a survey

Herrewijnen,

Nguyen,

Bex

et al. 2024

Front. Artif. Intell.

View full text Add to dashboard Cite

Asking annotators to explain “why” they labeled an instance yields annotator rationales: natural language explanations that provide reasons for classifications. In this work, we survey the collection and use of annotator rationales. Human-annotated rationales can improve data quality and form a valuable resource for improving machine learning models. Moreover, human-annotated rationales can inspire the construction and evaluation of model-annotated rationales, which can play an important role in explainable artificial intelligence.

show abstract

Section: E Ect On Annotation Timementioning

confidence: 99%

Human-annotated rationales and explainable text classification: a survey

Herrewijnen,

Nguyen,

Bex

et al. 2024

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…But the irrelevant, redundant, and noisy information captured by attentions makes it hard to find meaningful patterns. Alternatively, accumulation attentions that quantify how information flows across tokens can be used for interpretation (Abnar and Zuidema, 2020;Eberle et al, 2022). However, for casual LMs, information flows in one direction, and it causes an over-smoothing problem when the model is deep.…”

Section: Related Workmentioning

confidence: 99%

Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models

Hou,

Li,

Fei

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Recent work has shown that language models (LMs) have strong multi-step (i.e., procedural) reasoning capabilities. However, it is unclear whether LMs perform these tasks by cheating with answers memorized from pretraining corpus, or, via a multi-step reasoning mechanism. In this paper, we try to answer this question by exploring a mechanistic interpretation of LMs for multi-step reasoning tasks. Concretely, we hypothesize that the LM implicitly embeds a reasoning tree resembling the correct reasoning process within it. We test this hypothesis by introducing a new probing approach (called MechanisticProbe) that recovers the reasoning tree from the model's attention patterns. We use our probe to analyze two LMs: GPT-2 on a synthetic task (k-th smallest element), and LLaMA on two simple language-based reasoning tasks (ProofWriter & AI2 Reasoning Challenge). We show that MechanisticProbe is able to detect the information of the reasoning tree from the model's attentions for most examples, suggesting that the LM indeed is going through a process of multi-step reasoning within its architecture in many cases. 1

show abstract

“…In recent years, various attempts have been made to investigate the correlation of human attention with the machine attention of a pre-trained large language model (Eberle et al (2022), Sood et al (2020a), Bensemann et al (2022). Eberle et al (2022) highlighted the inability of cognitive models to account for the higher level cognitive activities like semantic role matching, hence motivating the use of large language models (LLMs) for modelling the human gaze. Hollenstein et al (2021) showed the efficacy of LLMs in predicting the gaze features for multiple languages, including English, Russian, Dutch and German.…”

Section: Gaze In Deep Learning-based Architecturesmentioning

confidence: 99%

Eyes Show the Way: Modelling Gaze Behaviour for Hallucination Detection

Maharaj,

Saxena,

Kumar

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Detecting hallucinations in natural language processing (NLP) is a critical undertaking that demands a deep understanding of both the semantic and pragmatic aspects of languages. Cognitive approaches that leverage users' behavioural signals, such as gaze, have demonstrated effectiveness in addressing NLP tasks with similar linguistic complexities. However, their potential in the context of hallucination detection remains largely unexplored. In this paper, we propose a novel cognitive approach for hallucination detection that leverages gaze signals from humans. We first collect and introduce an eye tracking corpus (IITB-HGC: IITB-Hallucination Gaze corpus) consisting of 500 instances, annotated by five annotators for hallucination detection. Our analysis reveals that humans selectively attend to relevant parts of the text based on distributional similarity, similar to the attention bias phenomenon in psychology. We identify two attention strategies employed by humans: global attention, which focuses on the most informative sentence, and local attention, which focuses on important words within a sentence. Leveraging these insights, we propose a novel cognitive framework for hallucination detection that incorporates these attention biases. Experimental evaluations on the FactCC dataset demonstrate the efficacy of our approach, obtaining a balanced accuracy of 87.1%. Our study highlights the potential of gaze-based approaches in addressing the task of hallucination detection and sheds light on the cognitive processes employed by humans in identifying inconsistencies.

show abstract

Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?

Cited by 7 publications

References 0 publications

Human-annotated rationales and explainable text classification: a survey

Human-annotated rationales and explainable text classification: a survey

Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models

Eyes Show the Way: Modelling Gaze Behaviour for Hallucination Detection

Contact Info

Product

Resources

About