Is a Phone-Based Language and Literacy Assessment a Reliable and Valid Measure of Children’s Reading Skills in Low-Resource Settings?

Abstract

Technology-based remote research methods are increasingly widespread, including learning assessments in child development and education research. However, little is known about whether technology-based remote assessments remain as valid and reliable as in-person assessments. We developed a low-cost phone-based language and literacy assessment for primary-school children in low-resource communities in rural Côte d’Ivoire using voice calls and SMS. We compared the reliability and validity of this phone-based assessment to an established in-person assessment.679 5th grade children completed language (phonological awareness, vocabulary, language comprehension) and literacy (letter, word, pseudoword, passage reading and comprehension) tasks in-person and by phone. Reliability (internal consistency) and predictive validity was high across in-person and phone-based tasks. In-person and phone-based assessments were moderately to strongly correlated. Phonological awareness and vocabulary skills measured in-person and by phone significantly predicted in-person and phone-based letter, word and pseudoword reading. Oral language and decoding skills measured in-person and by phone also significantly predicted in-person and phone-based passage reading and comprehension scores.Our phone-based assessment was a reliable and valid measure of language and reading and feasible for low-resource settings. Low-cost technologies offer significant potential to measure children’s learning remotely, contributing to greater inclusion of remote and low-resource populations in education research.

Variability in the age of schooling contributes to the link between literacy and numeracy in Côte d'Ivoire

“…One notable finding within the current work is the insignificant effect of vocabulary. In the current study, vocabulary was not predictive of literacy, contrary to previous work in Côte d'Ivoire using the same task (Sobers et al., 2023). The vocabulary measure, an antonym generation task, consisted of five items ( α = .61) and had comparable reliability to a 10‐item version (previously used with primary school children in Côte d'Ivoire ( α = .63)).…”

“…Vocabulary is a robust predictor of literacy in general (de Jong & van der Leij, 2002) and this specific task has been found to predict literacy in this population (Ball et al., 2022; Jasińska et al., 2022b). We investigated the reliability and validity of language and literacy tasks in this population in a separate manuscript (Sobers et al., 2023). We found the 10‐item version of the task was fairly reliable ( α = .63) and valid (i.e., correlated with other language measures and predicted reading performance); we also replicated the analysis with a five‐item version and found similar reliability and validity (see https://osf.io/wgsud/).…”

Does Nonlinguistic Segmentation Predict Literacy in Second Language Education? Statistical Learning in Ivorian Primary Schools

“…An ongoing literacy research program in Côte d'Ivoire was examining French literacy skills in primary school children, already including hundreds of children from dozens of schools, with follow‐up visits scheduled every 6 months from 2019 to 2022 (although COVID‐19 travel restrictions changed these procedures in 2020 and 2021; see Sobers et al., 2023). The students in this broader study were a randomly recruited pool of children in CM‐1 (the equivalent of fifth grade in the United States) classrooms across eight communities in the La Mé region of southeastern Côte d'Ivoire in 2019 and 2020.…”

Education without Formal Schooling through Tablets and Tutors: Evidence from Out-of-School Children in Bangladesh during the COVID-19 Pandemic

“…Learning over phone is still a new concept and there is much to be learned on the topic (Angrist et al 2020). In the recently developed literature on the use of phone-based education, most researchers have found positive results on student learning in Bangladesh (Wang et al 2023, Islam et al 2022, Hassan et al 2021, Kenya (Angrist et al 2023, Schueler et al 2022, Côte d'Ivoire (Sobers et al 2023), and India, Nepal, the Philippines, and Uganda (Angrist et al 2023), except for Crawfurd et al (2023) who find challenges in the implementation and spillover effects were affecting their results in Sierra Leone.…”