Using Eye-Tracking Measures to Predict Reading Comprehension

Abstract

Research on reading comprehension assessments suggests that they measure overlapping but not identical cognitive skills. In this paper, we examined the potential of eye-tracking as a tool for assessing reading comprehension. We administered three widely-used reading comprehension tests with varying task demands to 79 typical adult readers while monitoring their eye movements. In the York Assessment for Reading Comprehension (YARC), participants were given passages of text to read silently, followed by comprehension questions. In the Gray Oral Reading Test (GORT-5), participants were given passages of text to read aloud, followed by comprehension questions. In the sentence comprehension subtest of the Wide Range Achievement Test (WRAT-4), participants were given sentences with a missing word to read silently, and had to provide the missing word (i.e., a cloze task). Results from linear models predicting comprehension scores from eye-tracking measures yielded different patterns of results between the three tests. Models with eye-tracking measures always explained significantly more variance compared to baseline models with only reading speed, with R-squared 4 times higher for the YARC, 3 times for the GORT, and 1.3 times for the WRAT. Importantly, despite some similarities between the tests, no common good predictor of comprehension could be identified across the tests. Overall, the results suggest that reading comprehension tests do not measure the same cognitive skills to the same extent, and that participants adapted their reading strategies to the tests’ varying task demands. Finally, this study suggests that eye-tracking may provide a useful alternative for measuring reading comprehension.

Scanpath Regularity as an Index of Reading Comprehension

“…However, unlike other eye-movement measures, scanpath regularity assesses eye movements relative to the most common reading strategy employed to meet task demands -regardless of what those demands are. Thus, scanpath regularity may prove less susceptible to task-demand effects that influenced word-based measures in previous studies (e.g., Mézière et al, 2022), and hence may provide a more reliable predictor of reading comprehension per se, both as individual predictors and in combination with other eye-movement measures.…”

“…The data used in this study was initially collected for a study reported by Mézière et al (2022), which analyzed local and global eye-movements measures, but not scanpath regularity. For convenience we repeat the key points below.…”

“…The present study aims to investigate the relationship between eye-movement behavior and reading comprehension ability (Copeland et al, 2014(Copeland et al, , 2016Copeland & Gedeon, 2013;D'Mello et al, 2020;Inhoff et al, 2018;Martínez-Gómez & Aizawa, 2014;Mézière et al, 2022;Southwell et al, 2020) using two types of eye-tracking measures -scanpath regularity and word-based measures. Specifically, we investigate the relative contributions of these two types of measures as predictors of performance on reading comprehension tasks with varying task demands.…”

“…In the last decades, researchers have investigated eye movements as a potential measure of language and reading ability. Several studies have suggested that features of eye-movement behavior during reading such as fixation durations, saccade length, and regression rates could be used to estimate second language proficiency (Berzak et al, 2018;Reich et al, 2022), classify readers at risk of dyslexia (Benfatto et al, 2016;Rello & Ballesteros, 2015), as well as estimate performance on reading comprehension assessments (Copeland et al, 2014(Copeland et al, , 2016Copeland & Gedeon, 2013;D'Mello et al, 2020;Inhoff et al, 2018;Martínez-Gómez & Aizawa, 2014;Mézière et al, 2022;Reich et al, 2022;Southwell et al, 2020).…”

“…Studies on the relationship between eye movements and reading comprehension accuracy have shown that eye-movement behavior can successfully predict performance on commonly-used reading comprehension tasks, including cloze tasks (Copeland et al, 2014;Mézière et al, 2022), multiple-choice comprehension questions (D'Mello et al, 2020;Inhoff et al, 2018;Southwell et al, 2020), and open-ended questions (Mézière et al, 2022).…”