Auditing for Score Inflation Using Self-Monitoring Assessments: Findings From Three Pilot Studies

Koretz, Daniel; Jennings, Jennifer L.; Ng, Hui Leng; Yu, Carol; Braslow, David; Langi, Meredith

doi:10.1080/10627197.2016.1236674

Cited by 3 publications

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although some initial pilot testing suggests that an SMA design can identify some score inflation and show variations in inflation across categories of schools, (e.g., Koretz, Jennings, Ng, Yu, Braslow, & Langi, 2014), this approach, like Neal's, is largely untried and poses many unresolved issues. It remains unclear, for example, in what ways and to what degree audit items should differ from others to function well as audits while maintaining fidelity to the target of inference.…”

Section: Possibilities For Changes In Designmentioning

confidence: 99%

Adapting Educational Measurement to the Demands of Test-Based Accountability

Koretz¹

2015

Measurement: Interdisciplinary Research and Perspectives

Self Cite

View full text Add to dashboard Cite

Accountability has become a primary function of large-scale testing in the U.S.The pressure on educators to raise scores is vastly greater than it was several decades ago.Research has shown that high-stakes testing can generate behavioral responses that inflate scores, often severely. I argue that because of these responses, using tests for accountability necessitates major changes in the practices of educational measurement.The needed changes span the entire testing endeavor. This paper addresses implications for design, linking, and validation. It offers suggestions about possible new approaches and calls for research evaluating them.Adapting to accountability 1 Over the past several decades, accountability has become a primary functionarguably, the single most important function-of large-scale educational testing in the U.S. The transition has been gradual, dating back at least to the minimum-competency testing movement of the 1970s, and the nature of the accountability systems and the characteristics of the assessments used for this purpose have varied through several waves of policy initiatives (Koretz & Hamilton, 2006). Nonetheless, the pressure on educators to raise scores has increased markedly from one wave of initiatives to the next.The current situation, in which rewards and sanctions for schools based on scores are ubiquitous and consequential evaluations of teachers based on students' scores are in place or planned in a large number of states, represents a tremendous change from the typical uses of tests 40 years ago.The premise of this paper is that the current uses of tests for accountability require major changes to several aspects of educational measurement. The needed changes span the full sequence of measurement practices, beginning with test design, continuing with the activities needed to maintain testing programs, in particular, linking, and ending with validation.

show abstract

Section: Possibilities For Changes In Designmentioning

confidence: 99%

Adapting Educational Measurement to the Demands of Test-Based Accountability

Koretz¹

2015

Measurement: Interdisciplinary Research and Perspectives

Self Cite

View full text Add to dashboard Cite

show abstract

Accountability, Achievement, and Inequality in American Public Schools: A Review of the Literature

Mittleman

Jennings

2018

Handbooks of Sociology and Social Research

View full text Add to dashboard Cite

Improving Balance in Educational Measurement: A Legacy of E. F. Lindquist

Koretz

2024

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

A critically important balance in educational measurement between practical concerns and matters of technique has atrophied in recent decades, and as a result, some important issues in the field have not been adequately addressed. I start with the work of E. F. Lindquist, who exemplified the balance that is now wanting. Lindquist was arguably the most prolific developer of achievement tests in the history of the field and an accomplished statistician, but he nonetheless focused extensively on the practical limitations of testing and their implications for test development, test use, and inference. I describe the withering of this balance and discuss two pressing issues that have not been adequately addressed as a result: the lack of robustness of performance standards and score inflation. I conclude by discussing steps toward reestablishing the needed balance.

show abstract

Auditing for Score Inflation Using Self-Monitoring Assessments: Findings From Three Pilot Studies

Cited by 3 publications

References 22 publications

Adapting Educational Measurement to the Demands of Test-Based Accountability

Adapting Educational Measurement to the Demands of Test-Based Accountability

Accountability, Achievement, and Inequality in American Public Schools: A Review of the Literature

Improving Balance in Educational Measurement: A Legacy of E. F. Lindquist

Contact Info

Product

Resources

About