What Eye Movements Reveal About Later Comprehension of Long Connected Texts

Abstract

We know that reading involves coordination between textual characteristics and visual attention, but research linking eye movements during reading and comprehension assessed after reading is surprisingly limited, especially for reading long connected texts. We tested two competing possibilities: (a) the weak association hypothesis: Links between eye movements and comprehension are weak and short-lived, versus (b) the strong association hypothesis: The two are robustly linked, even after a delay. Using a predictive modeling approach, we trained regression models to predict comprehension scores from global eye movement features, using participant-level crossvalidation to ensure that the models generalize across participants. We used data from three studies in which readers (Ns = 104, 130, 147) answered multiple-choice comprehension questions 30 min after reading a 6,500-word text, or after reading up to eight 1,000-word texts. The models generated accurate predictions of participants' text comprehension scores (correlations between observed and predicted comprehension: 0.384, 0.362, 0.372, ps < .001), in line with the strong association hypothesis. We found that making more, but shorter fixations, consistently predicted comprehension across all studies. Furthermore, models trained on one study's data could successfully predict comprehension on the others, suggesting generalizability across studies. Collectively, these findings suggest that there is a robust link between eye movements and subsequent comprehension of a long connected text, thereby connecting theories of low-level eye movements with those of higher order text processing during reading.

Using measures of reading time regularity (RTR) to quantify eye movement dynamics, and how they are shaped by linguistic information

“…Moreover, they demonstrated that decoupling measured in instances of mind-wandering resulted in worse text comprehension. Moreover, Southwell and colleagues (2020) showed that comprehension scores can be successfully predicted from reading times and classical eye movement measures. However, it remains unclear why the same measures yielded null effects in other studies ( Wallot et al, 2015 ) or related reading speed components during self-paced reading ( LeVasseur et al, 2006 , 2008 ; Wallot et al, 2014 ).…”

Using Eye-Tracking Measures to Predict Reading Comprehension

“…In a recent study, Southwell, Gregg, Bixler, and D'Mello (2020) predicted reading comprehension scores in three datasets using seven global eye-movement measures (number of fixations, average fixation duration, mean saccade length, proportion of regressions, proportion of horizontal saccade, fixation dispersion, and reading time). In all three datasets, participants were given long passages of text to read silently, followed by multiple-choice questions.…”

Scanpath Regularity as an Index of Reading Comprehension

“…In the last decades, researchers have investigated eye movements as a potential measure of language and reading ability. Several studies have suggested that features of eye-movement behavior during reading such as fixation durations, saccade length, and regression rates could be used to estimate second language proficiency (Berzak et al, 2018;Reich et al, 2022), classify readers at risk of dyslexia (Benfatto et al, 2016;Rello & Ballesteros, 2015), as well as estimate performance on reading comprehension assessments (Copeland et al, 2014(Copeland et al, , 2016Copeland & Gedeon, 2013;D'Mello et al, 2020;Inhoff et al, 2018;Martínez-Gómez & Aizawa, 2014;Mézière et al, 2022;Reich et al, 2022;Southwell et al, 2020).…”

“…Studies on the relationship between eye movements and reading comprehension accuracy have shown that eye-movement behavior can successfully predict performance on commonly-used reading comprehension tasks, including cloze tasks (Copeland et al, 2014;Mézière et al, 2022), multiple-choice comprehension questions (D'Mello et al, 2020;Inhoff et al, 2018;Southwell et al, 2020), and open-ended questions (Mézière et al, 2022).…”

“…However, there are inconsistent findings regarding the types of predictive eye-movement behaviors. For example, studies have found that better reading comprehension is associated with fewer regressions (Kim et al, 2019;Parshina et al, 2021;Southwell et al, 2020), more regressions (Schotter, Tran, et al, 2014;Wonnacott et al, 2016), or not related to regressions at all (Christianson et al, 2017).…”