Long-Term Validity and Diagnostic Accuracy of a Reading Computer-Adaptive Test

Abstract

Universal screening is the first mechanism by which students are identified as at risk of failure in the context of multitiered systems of supports. This study examined the validity and diagnostic accuracy of a reading computer-adaptive test as a screener to identify state achievement test performance for third through fifth graders ( N = 1,696). Single time points and slopes within year and longitudinally were examined. Validity results for single points were moderate (.60–.79, p < .002). Validity for slopes and the state test were weak or not significant. Diagnostic accuracy cut scores that maximized sensitivity and specificity yielded high accuracy for single points whereas sensitivity was inadequate for slopes. Practical implications and future directions are presented.

Comparison of screening methods for computer adaptive tests to predict reading and math performance

“…In a comparison of CBM and CATs, SM was found to be a significant predictor of performance on a statewide test (Shapiro & Gebhardt, 2012). Expanding on Shapiro and Gebhardt (2012), Ochs et al (2016) investigated the diagnostic accuracy of SR and performance on a statewide achievement test. The results found that SR displayed moderate to high levels of diagnostic accuracy (Ochs et al, 2016).…”

“…Expanding on Shapiro and Gebhardt (2012), Ochs et al (2016) investigated the diagnostic accuracy of SR and performance on a statewide achievement test. The results found that SR displayed moderate to high levels of diagnostic accuracy (Ochs et al, 2016). Last, more recent research supports the validity of SR as a predictor of performance on a state test (Ochs et al, 2020).…”

“…Despite evidence supporting both SR and SM as significant predictors for proficiency in academic skills (Ochs et al, 2016(Ochs et al, , 2020Shapiro & Gebhardt, 2012), the literature is limited on the use of CATs as universal screening tools. Similarly, there is evidence for utilizing multiple measures to identify students for academic risk, but overall, the research is sparse and limited on utilizing two screeners or CATs in a gated screening framework to predict end-of-year proficiency.…”

“…The results found that SR displayed moderate to high levels of diagnostic accuracy (Ochs et al, 2016). Last, more recent research supports the validity of SR as a predictor of performance on a state test (Ochs et al, 2020). The literature supports the use of SR and SM as a predictor of end‐of‐year‐proficiency in reading and math.…”