At a Glance: The Impact of Gaze Aggregation Views on Syntactic Tagging

Klerke, Sigrid; Plank, Barbara

doi:10.18653/v1/d19-6408

Cited by 12 publications

(8 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The inductive bias of language processing models can be improved using the eye-tracking signal (Barrett et al, 2018;Klerke and Plank, 2019) and the modification leads to more "human-like" output in generative tasks (Takmaz et al, 2020;Sood et al, 2020b). This indicates that patterns of relative importance in computational representations 1 Our implementation adapts code from https:// pypi.org/project/textualheatmap/.…”

Section: Patterns Of Relative Importancementioning

confidence: 99%

Relative Importance in Sentence Processing

Hollenstein¹,

Beinborn²

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Determining the relative importance of the elements in a sentence is a key factor for effortless natural language understanding. For human language processing, we can approximate patterns of relative importance by measuring reading fixations using eye-tracking technology. In neural language models, gradientbased saliency methods indicate the relative importance of a token for the target objective. In this work, we compare patterns of relative importance in English language processing by humans and models and analyze the underlying linguistic patterns. We find that human processing patterns in English correlate strongly with saliency-based importance in language models and not with attention-based importance. Our results indicate that saliency could be a cognitively more plausible metric for interpreting neural language models. The code is available on github: https://github.com/ beinborn/relative_importance.

show abstract

Section: Patterns Of Relative Importancementioning

confidence: 99%

Relative Importance in Sentence Processing

Hollenstein¹,

Beinborn²

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…Here, English gaze data were used to improve POS induction for French. Klerke and Plank (2019) also found that predicting a gaze feature as an auxiliary task may help POS tagging a multitask learning setup.…”

Section: Using Gaze For Sequence Labelling and Sequence Classificationmentioning

confidence: 99%

“…Barrett et al (2016a), (2016b) showed that word‐type averages of gaze features helped POS induction better than token‐level features. Klerke and Plank (2019) found that word‐type variance was better than less aggregated gaze features. Using word‐type gaze features does not require gaze at test time.…”

Section: How To Use Gaze For Nlp?mentioning

confidence: 99%

“…Multitask learning combines what is learned about the main task with what is learned from the gaze signal using parameter sharing between the tasks. Predicting gaze as an auxiliary task in a multitask learning setup is yet another approach that leverages the gaze signal without needing gaze‐annotation of the main task data (González‐Garduño & Søgaard, 2017; Klerke et al, 2016; Klerke & Plank, 2019; Strzyz et al, 2019a). These studies employ a multitask learning setup for text compression, readability prediction, syntactic tagging and dependency parsing respectively, while also learning to predict one or more gaze features.…”

Section: How To Use Gaze For Nlp?mentioning

confidence: 99%

See 1 more Smart Citation

Sequence labelling and sequence classification with gaze: Novel uses of eye‐tracking data for Natural Language Processing

Barrett

Hollenstein

2020

Language and Linguist. Compass

View full text Add to dashboard Cite

Eye‐tracking data from reading provide a structured signal with a fine‐grained temporal resolution which closely follows the sequential structure of the text. It is highly correlated with the cognitive load associated with different stages of human, cognitive text processing. While eye‐tracking data have been extensively studied to understand human cognition, it has only recently been considered for Natural Language Processing (NLP). In this review, we provide a comprehensive overview of how gaze data are being used in data‐driven NLP, in particular for sequence labelling and sequence classification tasks. We argue that eye‐tracking may effectively counter one of the core challenges of machine‐learning‐based NLP: the scarcity of annotated data. We outline the recent advances in gaze‐augmented NLP to discuss how the gaze signal from human readers can be leveraged while also considering the potentials and limitations of this data source.

show abstract

“…Most previous work in gaze-supported NLP has used gaze as an input feature, e.g. for syntactic sequence labeling [36], classifying referential versus non-referential use of pronouns [82], reference resolution [30], key phrase extraction [86], or prediction of multi-word expressions [64]. Recently, Hollenstein et al [29] proposed to build a lexicon of gaze features given word types, overcoming the need for gaze data at test time.…”

Section: Gaze Integration In Neural Network Architecturesmentioning

confidence: 99%

Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention

Sood,

Tannert,

Mueller

et al. 2020

Preprint

View full text Add to dashboard Cite

A lack of corpora has so far limited advances in integrating human gaze data as a supervisory signal in neural attention mechanisms for natural language processing (NLP). We propose a novel hybrid text saliency model (TSM) that, for the first time, combines a cognitive model of reading with explicit human gaze supervision in a single machine learning framework. On four different corpora we demonstrate that our hybrid TSM duration predictions are highly correlated with human gaze ground truth. We further propose a novel joint modeling approach to integrate TSM predictions into the attention layer of a network designed for a specific upstream NLP task without the need for any task-specific human gaze data. We demonstrate that our joint model outperforms the state of the art in paraphrase generation on the Quora Question Pairs corpus by more than 10% in BLEU-4 and achieves state of the art performance for sentence compression on the challenging Google Sentence Compression corpus. As such, our work introduces a practical approach for bridging between data-driven and cognitive models and demonstrates a new way to integrate human gaze-guided neural attention into NLP tasks.34th Conference on Neural Information Processing Systems (NeurIPS 2020),

show abstract

At a Glance: The Impact of Gaze Aggregation Views on Syntactic Tagging

Cited by 12 publications

References 25 publications

Relative Importance in Sentence Processing

Relative Importance in Sentence Processing

Sequence labelling and sequence classification with gaze: Novel uses of eye‐tracking data for Natural Language Processing

Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention

Contact Info

Product

Resources

About