Limitations of Standard Scores in Individual Achievement Testing

Vertical achievement scales, which range from the lower elementary grades to high school, are used pervasively in educational assessment. Using simulated data modeled after real tests, the present article examines two procedures available for vertical scaling: a Thurstone method and three‐parameter item response theory. Neither procedure produced artifactual scale shrinkage; both procedures produced modest scale expansion for one simulated condition.

mentioning

confidence: 99%

Comparison of Item Response Theory and Thurstone Methods of Vertical Scaling

Yen¹,

Burket²

1997

“…The resulting scales were no more or less equal-interval than any other score scales because such properties can never be internally justified. On the other hand, one can infer that to Hoover (1984a), Clemans (1993), and Phillips and Clarizio (1988), the driving motivation for creating a vertical score scale using Thurstonian or IRT methods was to measure growth in a classical sense. Given this assumption, it is no surprise to observe their consternation over empirical findings that raised doubts about the plausibility of equal-interval scale properties.…”

Section: Discussionmentioning

confidence: 99%

“…To date, no satisfactory explanation has been given regarding the anomalous growth trends found on the tests that precipitated Hoover's critique in 1984. In hindsight, a remarkable aspect of the defense of vertical scaling offered by Yen and Burket in their publications was that at no point did they seem interested in arguing that the approach produces a scale with equal‐interval properties, even though this was the proposition at the crux of the critiques written by Hoover (1984a), Phillips and Clarizio (), and Clemans (). For example, while Yen et al (, ) responded quite forcefully to many of the specific elements of the Clemans critique, at no point did they respond to the central issue he had raised: when and under what conditions does a vertical scale have interval properties?…”

Section: Early Arguments Over Vertical Scale Interpretationsmentioning

confidence: 99%

“…Hoover was not alone in his skepticism about the interval properties of vertical score scales (see Camilli, Yamamoto, & Wang, 1993;Clemans, 1993;Phillips & Clarizio, 1988). A common thread to these critiques was the observation that the growth trends implied by the vertical scales under examination were counterintuitive.…”

Section: Early Arguments Over Vertical Scale Interpretationsmentioning

confidence: 99%

See 1 more Smart Citation

Measuring Growth With Vertical Scales

Briggs

2013

A vertical score scale is needed to measure growth across multiple tests in terms of absolute changes in magnitude. Since the warrant for subsequent growth interpretations depends upon the assumption that the scale has interval properties, the validation of a vertical scale would seem to require methods for distinguishing interval scales from ordinal scales. In taking up this issue, two different perspectives on educational measurement are contrasted: a metaphorical perspective and a classical perspective. Although the metaphorical perspective is more predominant, at present it provides no objective methods whereby the properties of a vertical scale can be validated. In contrast, when taking a classical perspective, the axioms of additive conjoint measurement can be used to test the hypothesis that the latent variable underlying a vertical scale is quantitative (supporting ratio or interval properties) rather than merely qualitative (supporting ordinal or nominal properties). The application of such an approach is illustrated with both a hypothetical example and by drawing upon recent research that has been conducted on the Lexile scale for reading comprehension.

“…The use of item response theory as a model for cognitive test data has recently introduced some controversial discrepancies concerning trends in the variability of mental traits with age (Burket, 1984;Clemans, 1993;Hoover, 1984aHoover, , 1984bHoover, , 1988Phillips & Clarizio, 1988a, 1988bYen, 1988). There is a strong, mutual reinforcement between the popular notion that variance of cognitive skills increases with age and the fact that grade equivalent and Thurstonian scales have traditionally confirmed this trend.…”

mentioning

confidence: 99%

Grade Equivalent and IRT Representations of Growth

Schulz¹,

Nicewander²

1997

It has long been a part of psychometric lore that the variance of children's scores on cognitive tests increases with age. This increasing‐variance phenomenon was first observed on Binet's intelligence measures in the early 1900s. An important detail in this matter is the fact that developmental scales based on age or grade have served as the medium for demonstrating the increasing‐variance phenomenon. Recently, developmental scales based on item response theory (IRT) have shown constant or decreasing variance of measures of achievement with increasing age. This discrepancy is o f practical and theoretical importance. Conclusions about the effects of variables on growth in achievement will depend on the metric chosen. In this study, growth in the mean of a latent educational achievement variable is assumed to be a negatively accelerated function o f grade; within‐grade variance is assumed to be constant across grade, and observed test scores are assumed to follow an IRT model. Under these assumptions, the variance of grade equivalent scores increases markedly. Perspective on this phenomenon is gained by examining longitudinal trends in centimeter and age equivalent measures of height.