Remote Testing of Reading Comprehension in 8-Year-Old Children: Mode and Setting Effects

Abstract

Proctored remote testing of cognitive abilities in the private homes of test-takers is becoming an increasingly popular alternative to standard psychological assessments in test centers or classrooms. Because these tests are administered under less standardized conditions, differences in computer devices or situational contexts might contribute to measurement biases that impede fair comparisons between test-takers. Because it is unclear whether cognitive remote testing might be a feasible assessment approach for young children, the present study ( N = 1,590) evaluated a test of reading comprehension administered to children at the age of 8 years. To disentangle mode from setting effects, the children finished the test either in the classroom on paper or computer or remotely on tablets or laptops. Analyses of differential response functioning found notable differences between assessment conditions for selected items. However, biases in test scores were largely negligible. Only for children with below-average reading comprehension small setting effects between on-site and remote testing were observed. Moreover, response effort was higher in the three computerized test versions, among which, reading on tablets most strongly resembled the paper condition. Overall, these results suggest that, on average, even for young children remote testing introduces little measurement bias.

Equivalence of Computer- and Paper-Based Administrations of the SCHNAPP Spelling Test in German for Six-Year-Old Children

“…Although paper-pencil assessments are still the de facto standard in many educational contexts, particularly for young children, digital procedures are increasingly adopted because they are often more economical, allow higher levels of standardization, and are more motivating for children (e.g., Gnambs & Lenhard, 2023; Wright, 2020). To address the demands of educational practice for screening instruments that can be routinely administered in schools, the recently developed SCHNAPP Spelling Test in German (Schöfl, Steinmair, Zepnik, Zehetner, et al, 2023) was created as a digital version to be administered on a tablet with digital pens and as a traditional paper-based version.…”

“…Moreover, in the digital test version children could work at their own pace and thus were not limited by either too slow or too quick item presentations. As a result, the SCHNAPP Spelling Test is one of rather few standardized competence tests in German for which equivalent paper and computer versions are available (see also Gnambs & Lenhard, 2023, for a German reading competence test). The availability of PBT and CBT versions of the test should facilitate its use in practice, for example, for large-scale screenings in school contexts.…”

“…Whether different assessment modes such as CBT or PBT systematically distort psychological measurements is still controversially discussed (e.g., Clinton, 2019; Gnambs & Lenhard, 2023; Wright, 2020; Zinn et al, 2021). Whereas early research, for example, on mode effects for tests of reading competence suggested that respondents achieved lower scores on the computer as compared to paper (e.g., Clinton, 2019), more recent studies found approximate measurement invariance between testing modes (e.g., Gnambs & Lenhard, 2023). Potential reasons for these contradictory findings are that mode effects (if they exist) are test-specific or depend on respondent characteristics such as computer familiarity (see Lynch, 2022).…”