Using Gaze to Predict Text Readability

González-Garduño, Ana Valeria; Søgaard, Anders

doi:10.18653/v1/w17-5050

Cited by 20 publications

(8 citation statements)

References 20 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, there has been some work in using eye-tracking to evaluate certain aspects of the text, like readability (Gonzalez-Garduño and Søgaard, 2017;Mishra et al, 2017), grammaticality (Klerke et al, 2015), etc.. Our work uses eyetracking to predict the score given by a reader to a complete piece of text (rather than just a sentence as done by Klerke et al (2015)) and show that the scoring is more reliable if the reader has understood the text.…”

Section: Related Workmentioning

confidence: 93%

Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour

Mathias¹,

Kanojia²,

Patel³

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Predicting a reader's rating of text quality is a challenging task that involves estimating different subjective aspects of the text, like structure, clarity, etc. Such subjective aspects are better handled using cognitive information. One such source of cognitive information is gaze behaviour. In this paper, we show that gaze behaviour does indeed help in effectively predicting the rating of text quality. To do this, we first model text quality as a function of three properties -organization, coherence and cohesion. Then, we demonstrate how capturing gaze behaviour helps in predicting each of these properties, and hence the overall quality, by reporting improvements obtained by adding gaze features to traditional textual features for score prediction. We also hypothesize that if a reader has fully understood the text, the corresponding gaze behaviour would give a better indication of the assigned rating, as opposed to partial understanding. Our experiments validate this hypothesis by showing greater agreement between the given rating and the predicted rating when the reader has a full understanding of the text.

show abstract

Section: Related Workmentioning

confidence: 93%

Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour

Mathias¹,

Kanojia²,

Patel³

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

show abstract

“…They found that the proportion of regressions to previously read text is sensitive to the differences in human-and computer-induced complexity. Gonzalez-Garduño and Søgaard (2017) show that text readability prediction improves significantly from hard parameter sharing when models try to predict word-based gaze features in a multi-task-learning setup. All of these works, however, use gaze data that was collected under laboratory conditions from skilled, adult readers.…”

Section: Related Workmentioning

confidence: 97%

“…Previous approaches that have used gaze data in the context of natural language processing include the work of Barrett et al (2016), who aim to improve part-of-speech induction with gaze features, Klerke et al (2016), where gaze data is used as an auxiliary task in sentence compression, and Klerke et al (2015b), where gaze data is used to evaluate the output of machine translation. The most related work is Klerke et al (2015a) and Gonzalez-Garduño and Søgaard (2017). Klerke et al (2015a) compared gaze from reading original, manually compressed, and automatically compressed sentences.…”

Section: Related Workmentioning

confidence: 99%

Predicting misreadings from gaze in children with reading difficulties

Bingel¹,

Barrett²,

Klerke³

2018

Proceedings of the Thirteenth Workshop on Innovative Use of NLP For Building Educational Applications

View full text Add to dashboard Cite

We present the first work on predicting reading mistakes in children with reading difficulties based on eye-tracking data from real-world reading teaching. Our approach employs several linguistic and gaze-based features to inform an ensemble of different classifiers, including multi-task learning models that let us transfer knowledge about individual readers to attain better predictions. Notably, the data we use in this work stems from noisy readings in the wild, outside of controlled lab conditions. Our experiments show that despite the noise and despite the small fraction of misreadings, gaze data improves the performance more than any other feature group and our models achieve good performance. We further show that gaze patterns for misread words do not fully generalize across readers, but that we can transfer some knowledge between readers using multitask learning at least in some cases. Applications of our models include partial automation of reading assessment as well as personalized text simplification.

show abstract

“…Using eye movement data to modify the inductive bias of language processing models has resulted in improvements for several NLP tasks (e.g., Barrett et al 2016;. It has also been used as a supervisory signal in multi-task learning scenarios (Klerke et al, 2016;Gonzalez-Garduno and Søgaard, 2017) and as a method to fine-tune the attention mechanism (Barrett et al, 2018). We use eye tracking data to evaluate how well transformer language models predict human sentence processing.…”

Section: Related Workmentioning

confidence: 99%

Multilingual Language Models Predict Human Reading Behavior

Hollenstein¹,

Pirovano²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

We analyze if large language models are able to predict patterns of human reading behavior. We compare the performance of language-specific and multilingual pretrained transformer models to predict reading time measures reflecting natural human sentence processing on Dutch, English, German, and Russian texts. This results in accurate models of human reading behavior, which indicates that transformer models implicitly encode relative importance in language in a way that is comparable to human processing mechanisms. We find that BERT and XLM models successfully predict a range of eye tracking features. In a series of experiments, we analyze the cross-domain and cross-language abilities of these models and show how they reflect human sentence processing.

show abstract

Using Gaze to Predict Text Readability

Cited by 20 publications

References 20 publications

Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour

Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour

Predicting misreadings from gaze in children with reading difficulties

Multilingual Language Models Predict Human Reading Behavior

Contact Info

Product

Resources

About