Setting and Validating Cut Scores for Tests

Pitoniak, Mary J.; Morgan, Deanna L.

doi:10.4324/9781315709307-21

Cited by 2 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We propose to apply results obtained using this methodology to the practice of standard-setting studies of pass/fail examinations, in particular, to include this information along with other impact data provided to the policymakers when they discuss the final decision on the cut-score. Pitoniak & Morgan (2012) write "the policymaker should take errors of classification into account, and reduce the type of error that deemed the most serious. "…”

Section: Introductionmentioning

confidence: 99%

Cut-Score Operating Function Extensions: Penalty-Based Errors and Uncertainty in Standard Settings

Grabovsky

Pace

Runyon

2021

Applied Psychological Measurement

View full text Add to dashboard Cite

We model pass/fail examinations aiming to provide a systematic tool to minimize classification errors. We use the method of cut-score operating functions to generate specific cut-scores on the basis of minimizing several important misclassification measures. The goal of this research is to examine the combined effects of a known distribution of examinee abilities and uncertainty in the standard setting on the optimal choice of the cut-score. In addition, we describe an online application that allows others to utilize the cut-score operating function for their own standard settings.

show abstract

Section: Introductionmentioning

confidence: 99%

Cut-Score Operating Function Extensions: Penalty-Based Errors and Uncertainty in Standard Settings

Grabovsky

Pace

Runyon

2021

Applied Psychological Measurement

View full text Add to dashboard Cite

show abstract

“…Consideration of negative consequences is particularly vital for standard setting. Pitoniak and Morgan (2018) insist that the collection of validity evidence is essential to setting performance standards. The decision to use any test with a standard must value judgments about the consequences of test use (AERA, APA, NCME, 2014).…”

Section: Introductionmentioning

confidence: 99%

“…Panels consist of subject matter experts representing institutions to which the decisions are being made (Pitoniak & Morgan, 2018). Panelists are considered highly regarded qualified experts in the field––a crucial quality for standard setting (Skorupski, 2012).…”

Section: Introductionmentioning

confidence: 99%

“…They have a high degree of expertise with the content to determine what knowledge a minimally qualified candidate can demonstrate on the examination. Standard setting facilitators train panelists to conceptualize the borderline examinee based on the performance‐level descriptors (Cizek & Earnest, 2016; Pitoniak & Morgan, 2018). For pass/fail examinations, the goal of the standard setting process is to decide on a score to separate the borderline competent candidate from the rest.…”

Section: Introductionmentioning

confidence: 99%

“…In test‐centered standard setting, two types of feedback information may be provided to panelists: (1) impact or (2) normative (Pitoniak & Morgan, 2018). Impact data are an indication of the consequences of panelist's recommended cut score.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Adding Objectivity to Standard Setting: Evaluating Consequence Using the Conscious and Subconscious Weight Methods

Leventhal

Grabovsky

2020

Educational Measurement

View full text Add to dashboard Cite

Standard setting is arguably one of the most subjective techniques in test development and psychometrics. The decisions when scores are compared to standards, however, are arguably the most consequential outcomes of testing. Providing licensure to practice in a profession has high stake consequences for the public. Denying graduation or forcing remediation has high‐impact consequences for students. Unfortunately, tests that classify individuals are subjected to false positive and false negative misclassifications. When determining a standard, standard setting panelists implicitly consider the negative consequences of the decisions made from test use. We propose the conscious weight method and subconscious weight method to bring more objectivity to the standard setting process. To do this, these methods quantify the relative harm of the negative consequences of false positive and false negative misclassification.

show abstract

Setting and Validating Cut Scores for Tests

Cited by 2 publications

References 0 publications

Cut-Score Operating Function Extensions: Penalty-Based Errors and Uncertainty in Standard Settings

Cut-Score Operating Function Extensions: Penalty-Based Errors and Uncertainty in Standard Settings

Adding Objectivity to Standard Setting: Evaluating Consequence Using the Conscious and Subconscious Weight Methods

Contact Info

Product

Resources

About