Test Design and Retrofit

Fulcher, Glenn

doi:10.1002/9781405198431.wbeal1199

Cited by 1 publication

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This has generated a significant amount of psychometric research that aims to retrofit norm-referenced language tests with criterionreferenced meaning, from rule-space methodology (Tatsuoka and Buck, 1998) to fusion theory (Jang, 2009), none of which has been successful when dealing with complex constructs such as language competence (Alderson, 2010). Providing substantive meaning to test scores requires validation by design, rather than validation by retrofit (Fulcher, 2013a). This assumes that we are able to provide descriptions of the domains to which test scores make predictions, and which provide the data for test design.…”

Section: The Corruption Criterionmentioning

confidence: 99%

Limited aspects of reality: Frames of reference in language assessment

Fulcher¹,

Svalberg²

2013

ijes

Self Cite

View full text Add to dashboard Cite

Language testers operate within two frames of reference: norm-referenced (NRT) and criterion-referenced testing (CRT). The former underpins the world of large-scale standardized testing that prioritizes variability and comparison. The latter supports substantive score meaning in formative and domain specific assessment. Some claim that the criterion-referenced enterprise is dead, save its legacy in score reporting (Davidson, 2012, p. 198). We argue that announcing the demise of CRT is premature. But we do acknowledge that what now passes as CRT is in fact not criterion-referenced, but is based upon a corruption of the original meaning of "criterion" as domain-specific performance. This distortion took place when NRT co-opted the term "standard" to serve as a rationale for the measurement enterprise of establishing cut-scores to retrofit NR tests with meaning derived from external scales. The true heirs of the CRT movement are researchers who base test design in the careful analysis of construct and content in domain specific communication.

show abstract

Section: The Corruption Criterionmentioning

confidence: 99%