An examination of alternate assessment durations when assessing multiple-skill computational fluency: The generalizability and dependability of curriculum-based outcomes within the context of educational decisions

Abstract

The current study extended previous research on curriculum-based measurement in mathematics (M-CBM) assessments. The purpose was to examine the generalizability and dependability of multiple-skill M-CBM computation assessments across various assessment durations (1, 2, 3, 4, 5, and 6 minutes). Results of generalizability and dependability studies (N ϭ 104 students) suggest that relative interindividual decisions can rely on the results from 1-minute administrations for low-stakes decisions and the results of 4-minute administrations for high-stakes decisions. Moreover, absolute intraindividual decisions can rely on the results from 4-minute administrations for low-stakes decisions and 13-minute administrations for high-stakes decisions. The implications and limitations of these results are discussed. © 2005 Wiley Periodicals, Inc.Curriculum-based measurement (CBM) is a standardized set of procedures that can be used to assess and evaluate academic performance in the basic skill areas of reading, spelling, mathematics, and written expression (Deno, 1985(Deno, , 2003Shinn, 2002Shinn, , 1989. The CBM literature in the area of mathematics (M-CBM) is replete with studies that demonstrate its utility for making educational decisions. Research supports the use of M-CBM for the purposes of progress monitoring (Allinder, 1996;Allinder & Beck, 1995;Stecker & Fuchs, 2000), modifying instruction for class groups (Fuchs, Fuchs, Hamlett, & Phillips, 1994;Whinnery & Fuchs, 1992), informing instructional groupings Fuchs, Fuchs, Bishop, & Hamlett, 1992;Spicuzza et al., 2001), modifying instruction for students with special needs (Fuchs, Fuchs, & Hamlett, 1989;Fuchs et al., 1994;Whinnery & Fuchs, 1992), identifying academic strengths and weaknesses (Fuchs et al., 1989;Fuchs, Fuchs, Hamlett, & Stecker, 1990), supporting peer-assisted learning (Calhoon & Fuchs, 2003), and predicting performance on statewide assessments (Helwig, Anderson, & Tindal, 2002).When CBM is used to assess computational fluency performance is quantified as the number of digits written correctly per unit of time. A review of the professional literature suggests that this metric for computational fluency is more sensitive to student growth than accuracy measures, which do not account for the rate of performance

Implications of Recent Research

“…Hence, the median score of three 2-min probes should be used when making decisions (Hintze et al, 2002). The results of some studies provide support for repeated 2-min administrations to derive the median level of performance (e.g., Hintze et al, 2002), whereas others support the use of extended administration durations (Christ, Johnson-Gros, & Hintze, 2005).…”

Development of curriculum-based measurements in mathematical computations for Arab-speaking fourth grade students

“…We also provided evidence for concurrent validity by showing high correlations between students' scores on the probes and their scores on the criterion measure of the MAIT (Christ et al., 2008; Shapiro et al., 2006; Thurber et al., 2002). Using three methods for testing reliability, namely test–retest (Christ et al., 2005), alternate format (Burns & Vanderhyden, 2006; Petscher, Cummings, Biancarosa, & Fien, 2010; Turber et al., 2002), and inter-rater reliability (Turber et al., 2002), the results showed that the measures are reliable and adequate for use by teachers and practitioners in the Omani context.…”

“…There is evidence supporting the use of MC-CBM for monitoring students' progress (Stecker, Fuchs, & Fuchs, 2005), modifying instruction for students with varying abilities (Slavin, & Lake, 2008), informing instructional groupings (McLeskey & Waldron, 2011), adapting instruction for students with disabilities (McMaster, Fuchs, Fuchs, & Compton, 2005), identifying students' academic strengths and weaknesses (Stecker, Fuchs, & Fuchs, 2005), supporting peer-assisted learning (Calhoon & Fuchs, 2003), and predicting students' performance on statewide assessments (Helwig, Anderson, & Tindal, 2002). The current study was informed by studies that examined performance indicators on MC-CBM to classify students (Burns, VanDerHeyden, & Jiban, 2006; VanDerHeyden & Burns, 2005), investigated the administration of MC-CBM on higher (Fuchs, Fuchs, Compton, Bryant, Hamlett, & Seethaler, 2007) compared to earlier grades (Clarke & Shinn, 2004; Hintze Christ, & Keller, 2002; Jitendra, Dupuis, & Zaslofsky, 2014; Shapiro, Dennis, & Fu, 2015; Thurber et al., 2002), and used MC-CBM to screen for students with LD in mathematics (Christ, Johnson-Gros, & Hintze, 2005; Fuchs et al, 2007; Hintze et al., 2002).…”

“…Although MC-CBM has gained robust empirical support, the findings concerning the psychometric properties of the MC-CBM used metric (i.e., digits correct per unit of time) and the assessment context (single-skill or multiple-skill) are inconclusive (Christ et al., 2005; Hintze et al., 2002; Methe, Briesch, and Hulac, 2015; Strait, Smith, Pender, Malone, Roberts, & Hall, 2015). Christ et al. (2005), for example, examined the generalizability and dependability of MC-CBM assessments across various assessment durations (1, 2, 3, 4, 5, and 6 minutes).…”

Generalizability of Universal Screening Measures for Behavioral and Emotional Risk

“…These can be conducted on various facets and are similar to the Spearman-Brown prophecy formula, which is used to determine how many items may be needed to obtain a reliable estimate of a trait (Cronbach et al, 1972). D studies have been employed to assess how dependability changes as a function of the length of probes (e.g., Christ, Johnson-Gros, & Hintze, 2005;Volpe et al, 2011) or the number of probes administered (e.g., Poncy, Skinner, & Axtell, 2005). Thus, a universal screening D study could examine the effect of increasing the number of occasions or ratings on the generalizability of the overall procedure (Webb, Shavelson, & Haertel, 2006).…”