Differences in Reading Screening Accuracy by Percentile Cutoff and English Proficiency: Feature Selection and Group-wise Prediction Evaluation

Abstract

This exploratory predictive study investigates our capacity to predict risk of developing reading difficulties based on a battery of tasks tapping into commonly used reading screening constructs. A key question pertains to the extent to which we can predict fairly and with equal accuracy for native English-speakers (EO) and emerging multilinguals (English learners [ELs] with sufficient English-proficiency to participate in English screening). This prediction exercise takes place in a linguistically diverse sample, over different prediction intervals (end of same and subsequent academic year), and across a range of percentile cut-offs (1st-25th) defining reading difficulty.Specifically, using a sample of 1,692 kindergarteners and first-graders from 24 schools across 13 Californian public school districts, we (a) identified the most important reading screening tasks for a range of percentile cut-offs defining ‘risk’ using random forest feature selection; (b) built a series of logisitic regression classifiers; and (c) evaluated these models’ performances for ELs and EO students spearately. When risk is defined as performance below the tenth percentile or higher, the same tasks generally surface as most important predictors of later word reading difficulty (letter naming and sounds for kindergarten; word and non-word reading for first graders), though with different relative importance for EO vs EL students and for the different prediction intervals. Though most models can be optimised to exceed sensitivity and specificity values of .80, classification accuracy is higher for EO students, especially for two-year predictions. Though models meeting conventionally used minimum accuracy requirements for both the EO and EL groups can be produced, more work is needed to achieve truly linguistically equitable (i.e., equally accurate) screening. We discuss the implications and feasibility of universal reading screening using English screening tasks in linguistically diverse populations.