Fixing a Broken Clerkship Assessment Process: Reflections on Objectivity and Equity Following the USMLE Step 1 Change to Pass/Fail

Abstract

Clerkship grading is a core feature of evaluation for medical students’ skills as physicians and is considered by most residency program directors to be an indicator of future performance and success. With the transition of the U.S. Medical Licensing Examination Step 1 score to pass/fail, there will likely be even greater reliance on clerkship grades, which raises several important issues that need to be urgently addressed. This article details the current landscape of clerkship grading and the systemic discrepancies in assessment and allocation of honors. The authors examine not only objectivity and fairness in clerkship grading but also the reliability of clerkship grading in predicting residency performance and the potential benefits and drawbacks to adoption of a pass/fail clinical clerkship grading system. In the promotion of a more fair and equitable residency selection process, there must be standardization of grading systems with consideration of explicit grading criteria, grading committees, and/or structured education of evaluators and assessors regarding implicit bias. In addition, greater adherence and enforcement of transparency in grade distributions in the Medical Student Performance Evaluation is needed. These changes have the potential to level the playing field, foster equitable comparisons, and ultimately add more fairness to the residency selection process.

How Automation and ChatGPT Will Affect Competency Definitions and Assessment Tools

“…To the Editor: Over 20years ago, the Accreditation Council for Graduate Medical Education defined medical knowledge as 1 of the 6 core competencies for physicians. 1 However, the wealth of medical information trainees are expected to learn has grown dramatically since that time. As technological progress ushers in the use of artificial intelligence (AI) chatbots for therapy, the use of language models (e.g., ChatGPT) for synthesizing information, and the use of automated transcription services for medical documentation, the competencies required by physicians are undergoing a paradigm shift.…”

“…To the Editor: With the rapid evolution of artificial intelligence (AI) and its growing influence in health care, the time is opportune to turn our focus toward the potential of advanced AI language models, like ChatGPT, in bringing about global equity in academic medicine. 1 The dialogue surrounding AI in health care is abundant but often overlooks the significant disparities in AI accessibility, particularly in underserved regions, thereby rendering it a pressing international concern.…”

Fairness Fixes: Two Strategies to Improve the Clerkship Assessment Process

“…To the Editor: We applaud the report by Huynh and colleagues on "Fixing a Broken Clerkship Assessment Process," 1 as there is a duty to be fair to society, students, and teachers-in that order. We call attention to 2 proven strategies that clerkship directors could implement now within the current system, before some of the article's authors reach their clerkships.…”