Novel Examination for Evaluating Medical Student Clinical Reasoning: Reliability and Association With Patients Seen

Hemmer, Paul A.; Dong, Ting; Durning, Steven J.; Pangaro, Louis N.

doi:10.7205/milmed-d-14-00576

Cited by 5 publications

(1 citation statement)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Exam points (30 % of the grade) are earned on three clerkship exams: the National Board of Medical Examiners (NBME) subject examination in medicine, and two locally developed examinations (a multiple-choice examination of commonly ordered tests, with fair internal consistency (median KR-20 of 0.5) and a videobased clinical reasoning exam (with a reliability of 0.74-0.83). 23 We transformed teacher-recommended grades and expert grade assignments into a point score, with Fail=0, Low Pass=2, Pass=4, High Pass=6, and Honors=8. This allowed us to compare teacher-recommended grades to the expert grade assignment using Pearson's correlation coefficient.…”

Section: Secondary Outcomesmentioning

confidence: 99%

Regular Formal Evaluation Sessions are Effective as Frame-of-Reference Training for Faculty Evaluators of Clerkship Medical Students

Hemmer

Dadekian²,

Terndrup

et al. 2015

J GEN INTERN MED

Self Cite

View full text Add to dashboard Cite

BACKGROUND: Face-to-face formal evaluation sessions between clerkship directors and faculty can facilitate the collection of trainee performance data and provide frame-of-reference training for faculty. OBJECTIVE: We hypothesized that ambulatory faculty who attended evaluation sessions at least once in an academic year (attendees) would use the Reporter-Interpreter-Manager/Educator (RIME) terminology more appropriately than faculty who did not attend evaluation sessions (non-attendees). DESIGN: Investigators conducted a retrospective cohort study using the narrative assessments of ambulatory internal medicine clerkship students during the 2008-2009 academic year. PARTICIPANTS: The study included assessments of 49 clerkship medical students, which comprised 293 individual teacher narratives. MAIN MEASURES: Single-teacher written and transcribed verbal comments about student performance were masked and reviewed by a panel of experts who, by consensus, (1) determined whether RIME was used, (2) counted the number of RIME utterances, and (3) assigned a grade based on the comments. Analysis included descriptive statistics and Pearson correlation coefficients. KEY RESULTS: The authors reviewed 293 individual teacher narratives regarding the performance of 49 students. Attendees explicitly used RIME more frequently than non-attendees (69.8 vs. 40.4 %; p< 0.0001). Grades recommended by attendees correlated more strongly with grades assigned by experts than grades recommended by non-attendees (r=0.72; 95 % CI (0.65, 0.78) vs. 0.47; 95 % CI (0.26, 0.64); p= 0.005). Grade recommendations from individual attendees and non-attendees each correlated significantly with overall student clerkship clinical performance [r=0.63; 95 % CI (0.54, 0.71) vs. 0.52 (0.36, 0.66), respectively], although the difference between the groups was not statistically significant (p=0.21). CONCLUSIONS: On an ambulatory clerkship, teachers who attended evaluation sessions used RIME terminology more frequently and provided more accurate grade recommendations than teachers who did not attend. Formal evaluation sessions may provide frame-of-reference training for the RIME framework, a method that improves the validity and reliability of workplace assessment.

show abstract

Section: Secondary Outcomesmentioning

confidence: 99%