Recurrent Issues and Recent Advances in Scoring Performance Assessments

Clauser, Brian E.

doi:10.1177/01466210022031778

Cited by 52 publications

(37 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Different versions of the DPA were employed, involving rating scales of different lengths and a different degree of decomposition into subtasks. The fine grained version involves a highly analytical judgment, whereas the global task area form requires a more holistic judgment (Clauser, 2000). The question can be raised whether both versions are equally decisive and informative about the state of the drivers' driving proficiency.…”

Section: Discussionmentioning

confidence: 99%

Validity of an On-Road Driver Performance Assessment Within an Initial Driver Training Context

Roelofs¹,

Vissers

Onna³

et al. 2009

Proceedings of the 5th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design : Dr

View full text Add to dashboard Cite

Summary: Based on a competence-oriented view of driving a driver performance assessment (DPA) has been developed to inform learner drivers about their progress in acquiring driving proficiency. An initial validation study was carried out to evaluate the adequacy of inferences based on DPA-scores assigned by driving instructors. The results of two pilots are presented. Implications for driving performance assessment within driver training programs are discussed.

show abstract

Section: Discussionmentioning

confidence: 99%

Validity of an On-Road Driver Performance Assessment Within an Initial Driver Training Context

Roelofs¹,

Vissers

Onna³

et al. 2009

Proceedings of the 5th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design : Dr

View full text Add to dashboard Cite

show abstract

“…Staff from both projects met informally over the years to exchange information. The NBME examination with automated scoring of Step 3 (Primum Computer Case Simulations) became operational in 2000 (P. Harik, personal communication, July 14, 2014), backed by a considerable body of research (Clauser 2000;Clauser et al 2002;Clyman et al 1995). 34 The system is currently owned by Measurement Incorporated.…”

Section: Automated Scoringmentioning

confidence: 99%

A Historical Survey of Research Regarding Constructed-Response Formats

Bejar

2017

Methodology of Educational Measurement and Assessment

View full text Add to dashboard Cite

This chapter chronicles ETS research and development contributions related to the use of constructed-response item formats. 1 The use of constructed responses in testing dates back to imperial China, where tests were used in the selection of civil servants. However, in the United States, the multiple-choice format became dominant during the twentieth century, following its invention and use by the SAT ® examinations created by the College Board in 1926. When ETS was created in 1947, post-secondary admissions testing was largely based on tests consisting of multiplechoice items. However, from the start, there were two camps at ETS: those who believed that multiple-choice tests were sufficiently adequate for the purpose of assessing "verbal" skills and those who believed that "direct" forms of assessment requiring written responses had a role to play. For constructed-response formats to regain a foothold in American education several hurdles would need to be overcome. Research at ETS was instrumental in overcoming those hurdles.The first hurdle was that of reliability, specifically the perennial issue of low interrater agreement, which plagued the acceptance of constructed-response formats for most of the twentieth century. The second hurdle was broadening the conception of validity to encompass more than predictive considerations, a process that began with the introduction of construct validity by Cronbach and Meehl (1955). Samuel Messick at ETS played a crucial role in this process by making construct validity relevant to educational tests. An inflexion point in the process of reincorporating constructed-response formats more widely in educational tests was marked 1 Constructed responses to a prompt or question can range in scope and complexity. Perhaps the most common constructed response is the written essay. However, short written responses to questions are also considered to be constructed, as are spoken answers in response to a prompt, mathematical responses (equations, plotted functions, etc.), computer programs, and graphical responses such as architectural designs.

show abstract

“…In contrast to traditional forms of assessment, performance assessments are not standardized tests. This implies that traditional reliability indices may not be suitable to apply to performance assessments (Clauser, 2000;Dochy, 2009). Kane (1996) points out that the precision of measurements is broader than just reliability.…”

Section: Introductionmentioning

confidence: 99%

Towards an Integrative Formative Approach of Data-Driven Decision Making, Assessment for Learning, and Diagnostic Testing

Hubregtse¹,

Eggen²

2012

Psychometrics in Practice at RCEC

View full text Add to dashboard Cite

Classification accuracy of single exams is well studied in the educational measurement literature. However, when making important decisions, such as certification decisions, one usually uses several exams: an exam set. This chapter elaborates on classification accuracy of exam sets. This is influenced by the shape of the ability distribution, the height of the standards, and the possibility for compensation. This is studied using an example from vocational education and training (VET). The classification accuracy for an exam set is computed using item response theory (IRT) simulation. Classification accuracy is high when all exams from an exam set have equal and standardized ability distributions. Furthermore, exams where few or no students pass or fail increase classification accuracy. Finally, allowing compensation increases classification accuracy.

show abstract

Recurrent Issues and Recent Advances in Scoring Performance Assessments

Cited by 52 publications

References 40 publications

Validity of an On-Road Driver Performance Assessment Within an Initial Driver Training Context

Validity of an On-Road Driver Performance Assessment Within an Initial Driver Training Context

A Historical Survey of Research Regarding Constructed-Response Formats

Towards an Integrative Formative Approach of Data-Driven Decision Making, Assessment for Learning, and Diagnostic Testing

Contact Info

Product

Resources

About