Predictive Utility and Classification Accuracy of Oral Reading Fluency and the Measures of Academic Progress for the Wisconsin Knowledge and Concepts Exam

Abstract

This study examined the predictive validity and classification accuracy of two commonly used universal screening measures relative to a statewide achievement test. Results indicated that second-grade performance on oral reading fluency and the Measures of Academic Progress (MAP), together with special education status, explained 68% of the variance in third-grade standardized test performance (N = 399). Six methods of dichotomous risk classification were also explored to predict whether students would pass or fail the state test. A random sample of half of the participants was used to establish locally appropriate risk cut-scores, holding sensitivity constant at .95; the cut-scores were then cross-validated on the remaining participants. Cross-validation reflected sensitivity of above .90 for all methods, with MAP Language Usage and two multi-screener methods demonstrating the strongest overall technical adequacy for risk classification. The limitations and implications of these findings are discussed.

Evaluating the Criterion Validity and Classification Accuracy of Universal Screening Measures in Reading

“…Generally, the findings from the current study were similar to or higher than the range of correlations reported in previous studies for predictive validity using either MAP or state assessments as the criterion measure (Ball & O'Connor, 2016;Klingbeil et al, 2015;. Additionally, the recent study conducted by MAP test publishers corresponded precisely with the concurrent validity findings from the first cohort in the present study (NWEA, 2016).…”

“…Strong associations between MAP and the state assessment corroborate previous research of strong associations between MAP and other standardized assessments of reading achievement (Ball & O'Connor, 2016;January & Ardoin, 2015;Klingbeil et al, 2015Klingbeil et al, , 2017. However, the concurrent relation between MAP and the standardized assessment was slightly lower than findings from the one previous study that evaluated concurrent validity with a norm-referenced assessment (January & Ardoin, 2015).…”

“…Next, hierarchical linear regression was used to determine the shared and unique variance of screening measures in relation to the Georgia Milestones. MAP was entered first because it shares the characteristic of being a computer-administered assessment, is convenient to employ, is supported by an emerging peer-reviewed literature base (e.g., Ball & O'Connor, 2016;January & Ardoin, 2015;Klingbeil et al, 2017), and has content overlap with the Georgia Milestones.…”

“…An accumulation of research also supports the diagnostic qualities of CBM-R as a valid predictor of student risk for reading difficulties (Kilgus, Methe, Maggin, & Tomasula, 2014;Yeo, 2010 However, results of this study are limited to the specific criterion measure used (i.e., state test in Wisconsin) and the homogenous sample of students with respect to race/ethnicity and socioeconomic status. Additionally, the locally derived cut-scores may not generalize to lower performing students given the low base rates (11%) of failing the state test (Ball & O'Connor, 2016). Research is needed with samples of differing demographic characteristics, such as those students of racial/ethnic minority background and who live in low-resource communities.…”

Special Education Teachers’ Identification of Students’ Reading Difficulties in Grade 6

“…A high-quality screening measure should be able to accurately identify those students who are having RD as well as those whose development is proceeding according to expectations (Ball & O'Connor, 2016). Of these two aspects, the sensitivity of the assessment tool (i.e., its accuracy in identifying students with problems) has usually been considered important so that support can be allocated to those students who need it most.…”