Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study

Abstract

Purpose: This study aims to apply the yes/no Angoff and Hofstee methods to actual Korean Medical Licensing Examination (KMLE) 2022 written examination data to estimate cut scores for the written KMLE.Methods: Fourteen panelists gathered to derive the cut score of the 86th KMLE written examination data using the yes/no Angoff method. The panel reviewed the items individually before the meeting and shared their respective understanding of the minimum-compatibility physician. The standard setting process was conducted in 5 rounds over a total of 800 minutes. In addition, 2 rounds of the Hofstee method were conducted before starting the standard setting process and after the second round of yes/no Angoff.Results: For yes/no Angoff, as each round progressed, the panel’s opinion gradually converged to a cut score of 198 points, and the final passing rate was 95.1%. The Hofstee cut score was 208 points out of a maximum 320 with a passing rate of 92.1% at the first round. It scored 204 points with a passing rate of 93.3% in the second round.Conclusion: The difference between the cut scores obtained through yes/no Angoff and Hofstee methods did not exceed 2% points, and they were within the range of cut scores from previous studies. In both methods, the difference between the panelists decreased as rounds were repeated. Overall, our findings suggest the acceptability of cut scores and the possibility of independent use of both methods.

Issues in the 3rd year of the COVID-19 pandemic, including computer-based testing, study design, ChatGPT, journal metrics, and appreciation to reviewers

“…First, the standard setting of CBT still has not been adopted, although there were some studies on the cut score in the 2022 volume of the Journal of Educational Evaluation for Health Professions (JEEHP). Kim et al [ 2 ] suggested that acceptable cut scores for the KMLE could be derived and proposed independently using the yes/no Angoff and Hostee methods. Park [ 3 ] recommended using the yes/no Angoff method with a 60% probability for deciding “yes” as a substitute for the percent Angoff method in estimating the cutoff score of the KMLE.…”

Presidential address: improving item validity and adopting computer-based testing, clinical skills assessments, artificial intelligence, and virtual reality in health professions licensing examinations in Korea

“…The fixed passing score may cause variability in the proportion of passed examinees regardless of the examinees’ minimum requirements. The yes/no Angoff and Hofstee methods have been suggested for the Korean Medical Licensing Examination [ 12 ]. A revision of the law is required to adopt those standard-setting proposals, which is also an urgent task of KHPLEI.…”