Accuracy of Medical Student Electronic Logbook Problem List Entry

Abstract

Under ideal study circumstances, there was significant underreporting of core problems by students. Although the high specificity, meaning that students are not reporting problems they have not encountered, is reassuring, logbook sensitivity in this study was not good enough for high-stakes evaluations of students or for medical school licensing.

Internal Medicine Core Clerkships Experience With Core Problem Lists: Results From a National Survey of Clerkship Directors in Internal Medicine

“…In three recent studies, students missed 40%, 42%, and 49% of the core problems that should have been recorded but did not substantially overreport core problems that were not present (5%, 7%, and 15%, respectively). [5][6][7] Further, a fourth study showed 77% concordance between students' recorded principal diagnosis and faculty review, which rose to 97% when considering principal or secondary diagnoses. 8 In other words, the problems that students are recording in logbooks represent what was truly present, but students are not necessarily recording all problems that were present.…”

“…If clerkship directors are verifying the accuracy of medical student entry, recent educational literature would indicate that this is not necessary [5][6][7] if individual schools' logbooks perform at the same level as reports in the literature. In three recent studies, students missed 40%, 42%, and 49% of the core problems that should have been recorded but did not substantially overreport core problems that were not present (5%, 7%, and 15%, respectively).…”

“…3,4 Most recent reports on the accuracy of medical student problem lists entered into logbooks suggest that students miss a substantial proportion of the problems they encounter but do not substantially overreport problems that are not present. 5,6 In other words, the problems that a student reports are usually present, though students may not report every problem that was present. This would seem to indicate checking the accuracy of problem lists may not be necessary, unless clerkship directors were concerned with completeness of reporting.…”

Novel Examination for Evaluating Medical Student Clinical Reasoning: Reliability and Association With Patients Seen

“…9 In keeping with the principles o f deliberate practice put forth by E ricsson et a l,10 there is some evidence linking a higher volum e o f patients seen and the quality o f care with success in the board certification p ro cess.11' 12 At the clerk ship level, an investigation found a positive relationship betw een num bers o f patients seen and faculty judgm ents of medical student achievement13; there have been studies both supporting and refuting an association between the numbers of patients seen on a clerkship and end-of-clerkship assess ments such as National Board of Medical Examiners subject examinations, although the general trend supports such an association.14"16 However, to date, there has not been a study exploring the relationship between the clinical problems med ical students report seeing on a clinical clerkship and perfor mance on an assessment of clinical reasoning. In this study, we report validity evidence,17 to include measures of reliability and association with other variables, for a novel assessment of clinical reasoning, the Multistep Examination of Analytic Ability (MSX), which is based on the presumption of a slow, methodic process to analytic reasoning that can be dissected into discrete steps.…”

Relationship between clinical experiences and internal medicine clerkship performance

“…Students in most medical schools and internal medicine clerkships are required to maintain lists of their patient experiences and exposure to core problems in the discipline in a logbook 2 . Previous studies have reported variable accuracy in logbooks, usually involving significant under‐reporting, but little over‐reporting 2–4 . In other words, students do not seem to overestimate exposure to core problems and patients, but they may not report every instance of an exposure.…”