Technical Adequacy of General Outcome Measures for Middle School Mathematics

Abstract

This study examined three aspects of technical adequacy with respect to two general outcome measures in mathematics. The first two aspects were the effects of aggregating scores and correcting for random guessing on the reliability and validity of the measures. The third issue was the extent to which the measures were sensitive to changes in student performance. One hundred and five sixth-grade students completed two general outcome measures (basic facts and estimation) weekly for a 10-week period. With respect to aggregation, the results indicated that while fluency with basic facts is measured quite precisely with a single score, aggregation of scores for the estimation task increases both reliability and validity. Correcting the estimation scores for random guessing is recommended to increase criterion validity. Analyses of sensitivity to growth revealed that although both the fact and estimation measures reflect statistically significant changes across the 10-week period, the fact measure is likely to have greater practical significance. The article closes with a discussion of the relative merits of the measures for special education practice.

Criterion‐related validity of two curriculum‐based measures of mathematical skill in relation to reading comprehension in secondary students

“…In a series of studies, these researchers documented adequate reliability in respect to internal consistency, test–retest, and parallel forms. Criterion validity analyses found that low to moderate relations existed between students’ computation M‐CBM scores and classroom grades, standardized test scores, and teacher ratings, with coefficients ranging from .29 to .66 (Foegen, , ; Foegen & Deno, ). More recently, researchers used latent factor analysis to examine the predictive validity of several CBM measures to a high‐stakes state test.…”

CBM reading, mathematics, and written expression at the secondary level: Examining latent composite relations among indices and unique predictions with a state achievement test.

“…Computation and application aspects of mathematics, as used in CBM, have been shown to be highly related but represent distinct constructs (Thurber, Shinn, & Smolkowski, 2002). Basic fact fluency may represent a robust indictor of mathematics performance with utility for treatment planning (Christ, Scullin, Tolbize, & Jiban, 2008; Espin, Deno, Maruyama, & Cohen, 1989; Foegen, 2000; Foegen & Deno, 2001), albeit it is unlikely to be representative of general mathematics achievement (Christ et al, 2008). Computation fluency is considered the hallmark of mathematics learning disabilities (Gersten, Jordan, & Flojo, 2005), and neuroscientific evidence supports that fact fluency is associated with performance on higher level mathematics tasks (e.g., Price, Mazzocco, & Ansari, 2013).…”

Response to intervention in mathematics: Critical elements

“…One type of measurement that has been widely used in RtI frameworks for universal screening is Curriculum‐Based Measurement (CBM; Deno, 1985). In the area of mathematics, CBM universal screeners are available for pre‐K to first grades in early numeracy, for elementary students in computation and concepts and applications, and for secondary students in estimation and algebra (Chard et al, 2005; Clarke & Shinn, 2004; Foegen, 2000; Foegen, Olson, & Impecoven‐Lind, 2008; Fuchs, Hamlett, & Fuchs, 1998, 1999; Lembke & Foegen, 2009). The screening tools chart on the National Center for RtI Web site (rti4success.org), which is updated annually, provides an expert evaluation of screening tools that are submitted for examination.…”