Some Problems, Pitfalls, and Paradoxes in Educational Measurement

Brennan, Robert L.

doi:10.1111/j.1745-3992.2001.tb00071.x

Cited by 25 publications

(14 citation statements)

References 38 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reliability of a difference score is generally considered lower than the reliability of any single test (Brennan, 2001;Lord, 1958;Overall & Woodward, 1975;Payne & Jones, 1957;Schulte & Borich, 1984;Sternberg & Grigorenko, 2002;Zimmerman, 1994). The reliability of an intelligence-achievement difference score can be calculated using the formula:…”

Section: Measurement Errormentioning

confidence: 99%

Measurement error: implications for diagnosis and discrepancy models of developmental dyslexia

Cotton

Crewther

2005

Dyslexia

View full text Add to dashboard Cite

The diagnosis of developmental dyslexia (DD) is reliant on a discrepancy between intellectual functioning and reading achievement. Discrepancy-based formulae have frequently been employed to establish the significance of the difference between 'intelligence' and 'actual' reading achievement. These formulae, however, often fail to take into consideration test reliability and the error associated with a single test score. This paper provides an illustration of the potential effects that test reliability and measurement error can have on the diagnosis of dyslexia, with particular reference to discrepancy models. The roles of reliability and standard error of measurement (SEM) in classic test theory are also briefly reviewed. This is followed by illustrations of how SEM and test reliability can aid with the interpretation of a simple discrepancy-based formula of DD. It is proposed that a lack of consideration of test theory in the use of discrepancy-based models of DD can lead to misdiagnosis (both false positives and false negatives). Further, misdiagnosis in research samples affects reproducibility and generalizability of findings. This in turn, may explain current inconsistencies in research on the perceptual, sensory, and motor correlates of dyslexia.

show abstract

Section: Measurement Errormentioning

confidence: 99%

Measurement error: implications for diagnosis and discrepancy models of developmental dyslexia

Cotton

Crewther

2005

Dyslexia

View full text Add to dashboard Cite

show abstract

“…It is also the potential consequences that generate the controversysurrounding these tests, with the proponents (Cizek, 2001;Ward, 2000) looking forward to a wide range of positive consequences, and the critics (Brennan, 2001;Ehrenfeld, 2001;Eisner, 2001;Galley, 2001;Haney, 1991;Jones, 2001;Stoskopf, 2001;Thompson, 2001) fearing a rash of negative consequences. There does not seem to be much disagreement about the legitimacy of using the actual or anticipated consequences of the testing program in arguments for or against such testing programs.…”

mentioning

confidence: 99%

Validating High‐Stakes Testing Programs

Kane

2002

Educational Measurement

106

View full text Add to dashboard Cite

Cronbach made the point that for validity arguments to be convincing to diverse stakeholders, they need to be based on assumptions that are credible to these stakeholders. The interpretations and uses of high‐stakes test scores rely on a number of policy assumptions about what should be taught in schools, and more specifically, about the content standards and performance standards that should be applied to students and schools. For example, a high‐school graduation test can be developed as a measure of readiness for the world of work, for college, or for citizenship and the activities of daily life. The assumptions built into the assessment need to be subjected to scrutiny and criticism if a strong case is to be made for the validity of the proposed interpretation and use.

show abstract

“…Their reasoning is partly based on the fact that hardly any existing method could satisfy this criterion (Brennan, 2001). Nevertheless, it should be noted that in those methods a single equating transformation is used for the chosen population.…”

Section: The Criterion Of Equitymentioning

confidence: 99%

Local Equating of Cognitively Diagnostic Modeled Observed Scores

Xin

Zhang

2014

Applied Psychological Measurement

View full text Add to dashboard Cite

A local equating method of cognitively diagnostic modeled observed score was introduced to address the problem of equating without anchor test as well as the satisfaction of equating criteria. The criterion of test construct was redefined on attribute level. Then, a true family of equating transformations given attribute mastery pattern was derived from the criterion of equity, which is the main concern of local equating. The results of simulation studies showed a substantial bias for the traditional equipercentile equating under the equivalent groups design but much smaller bias for the local method. The proposed method also showed satisfactory performances under the nonequivalent groups design even without anchor test. The local method appeared to be quite robust with respect to changes in the sample size, but its biases became larger with increasing model misfit.

show abstract

Some Problems, Pitfalls, and Paradoxes in Educational Measurement

Cited by 25 publications

References 38 publications

Measurement error: implications for diagnosis and discrepancy models of developmental dyslexia

Measurement error: implications for diagnosis and discrepancy models of developmental dyslexia

Validating High‐Stakes Testing Programs

Local Equating of Cognitively Diagnostic Modeled Observed Scores

Contact Info

Product

Resources

About