Evaluation of schedule frequency and density when monitoring progress with curriculum-based measurement.

Abstract

School-based professionals often use curriculum-based measurement of reading (CBM-R) to monitor the progress of students with reading difficulties. Much of the extant CBM-R progress monitoring research has focused on its use for making group-level decisions, and less is known about using CBM-R to make decisions at the individual level. To inform the administration and use of CBM-R progress monitoring data, the current study evaluated the utility of 4 progress monitoring schedules that differed in frequency (once or twice weekly) and density (1 or 3 probes). Participants included 79 students (43% female; 51% White, 25% Hispanic or Latino, 11% Black or African American, 1% other, 12% unknown) in Grades 2 (n ϭ 45) and 4 (n ϭ 34) who were monitored across 10 weeks (February to May). Consistent with a focus on individual-level decision making, we used regression and mixed-factorial analysis of variances (ANOVAs) to evaluate the effect of progress monitoring schedule frequency, schedule density, grade level, and their interaction effects on CBM-R intercept, slope, SE of the slope (SEb), and SE of the estimate (SEE). Results revealed that (a) progress monitoring schedule frequency and density influenced the magnitude of SEb, (b) density had a significant but negligible impact on SEE, and (c) grade level had a significant effect on slope and intercept. None of the interaction effects were statistically significant. Findings from this study have implications for practitioners and researchers aiming to monitor students' progress with CBM-R. Impact and ImplicationsFindings of this study suggest that when monitoring the progress of students in Grades 2 and 4 with curriculum-based measures of oral reading rate, the number of times data are collected per week and the number of passages collected per occasion influence the technical characteristics of the resulting data, which may impact interpretations of students' response to instruction. Researchers and practitioners using curriculum-based measures of oral reading rate to monitor students' progress in 2nd and 4th grade should consider collecting data twice weekly and/or administering three passages per session to increase the precision of slope estimates.

Building an intelligent recommendation system for personalized test scheduling in computerized assessments: A reinforcement learning approach

“…On the other hand, in the context of progress monitoring, more frequent testing has been praised for increasing the precision of students' academic growth estimation by reducing the measurement error (Mellard et al, 2009;Christ et al, 2012). For example, January et al (2019) investigated the impact of testing frequency and the density of progress monitoring schedules on the accuracy of performance growth estimation in reading for second and fourth graders. The findings indicated that assessing students more frequently (e.g., twice a week rather than once a week) could significantly improve the confidence of accurately measuring students' academic growth.…”

“…More recently, a few studies have explored the possibility of utilizing big data in education to provide individualized recommendations for test administration in computerized formative assessments (Bulut et al, 2020). The studies focused on providing systematic help to the teachers, to increase the capacity to individually monitor, evaluate, and make decisions based on students' learning trajectory (January et al, 2019;Dede, 2016;Fischer et al, 2020).…”

An Intelligent Recommender System for Personalized Test Administration Scheduling With Computerized Formative Assessments

“…However, this view fails to consider the effect of testing on the individual student (i.e., testing fatigue or burn-out) and the effect of the broader educational context (e.g., missing instructional time or wasted resources). Also, many researchers have noted the lack of consensus on the optimal number of test administrations or the testing frequency that should be used with computerized formative assessments (e.g., Nelson et al, 2017;January et al, 2018January et al, , 2019Van Norman and Ysseldyke, 2020). This is due, at least in part, to the fact that the optimal number and frequency of test administrations depends on many factors such as grade level, subject (e.g., reading or mathematics), the type of computerized assessment (e.g., adaptive or non-adaptive assessment), and individual students' response to instruction.…”

“…Currently, one of the major challenges for school-based professionals is determining the timing and frequency of test administrations. For some time, researchers have argued that frequent test administrations over a long period can be highly beneficial when making individuallevel decisions based on formative assessments (Christ et al, 2012;Thornblad and Christ, 2014;January et al, 2019). The potential issue with this approach, however, is that frequent testing (e.g., weekly or bi-weekly) diminishes the amount of instructional time that students receive, which may exacerbate some of their difficulties in further developing their academic skills.…”

Making the Most of Community-Based Instruction: Progress Monitoring for Strategic Decision Making

“…Ideally, data should be collected at least once during every instructional session, although this does not mean that data need to be collected on every student response during that session (Test et al, 2017). However, evidence from progress-monitoring research on academic skills suggests that more frequent data collection allows educators to identify trends in data with greater precision, which may support more effective instructional decision making (January et al, 2019). In other words, it is important for educators to collect data frequently so that they can make timely decisions regarding the effectiveness of their instructional approaches.…”