Assessment of Professionalism During the Emergency Medicine Clerkship Using the National Clinical Assessment Tool for Medical Students in Emergency Medicine

Abstract

Objectives: In 2016, a national consensus conference created the National Clinical Assessment Tool for Medical Students in Emergency Medicine (NCAT-EM), a standardized end-of-shift assessment tool. We report the first large-scale analysis of professionalism concerns collected from May 2017 through December 2018 by a multisite consortium using the NCAT-EM. Our primary objective was to characterize the nature and frequency of professionalism concerns. Our secondary objective was to identify characteristics associated with giving or receiving a professionalism flag. Methods:The consortium database includes assessments for all students on EM clerkships at participating sites. This report presents descriptive statistics about the frequency of different flags, the distribution of flags among different student categories, assessor and student characteristics, and distribution of global assessment scores on assessments citing concerns. We used Fisher's exact test to look for associations between the frequency of professionalism flags and the sex of the students and assessors and across student categories. We used logistic regression to look for relationships between professionalism concerns and global assessment scores as well as intent to apply in EM.Results: We screened 6,768 assessments of 784 students by 719 assessors from 13 sites. After excluding assessments without flags and assessments with apparent data entry errors, we analyzed 57 (0.8%) assessments containing 79 flags. The most frequent flags were punctuality (25/79, 31.6%) and initiative (20/79, 25.3%). Few students received flags (42/784, 5.4%). Few assessors flagged concerns (41/719, 5.7%). We detected no correlation between the frequency of flags and whether a student was applying in EM or between the sex of students and assessors. Global scores of lower one-third appeared more often in assessments with a flag (30/57, 52.6% vs. 233/6,711, 3.5%).Conclusions: Only 5.4% of students received flags. Punctuality and initiative accounted for a majority of citations. Professionalism flags correlated strongly with lower global assessment scores.

Multi‐institutional Implementation of the National Clinical Assessment Tool in Emergency Medicine: Data From the First Year of Use

“…The mean rating scale score for students with no professionalism concerns was 69.7%, compared to 55.2% for students with one concern and 42.8% for students with two or more concerns. Professionalism data are analyzed in greater detail elsewhere 21 …”

“…Professionalism data are analyzed in greater detail elsewhere. 21 Global assessment data were available for 94.5% of all assessments. There was a positive skew, with only 256 (4.0%) of all students rated at the "lower third" level.…”

Assessing Individual Competency Differences Between Third- and Fourth-Year Medical Students Using the National Clinical Assessment Tool for Medical Students in the Emergency Department

“…They found that there was an overall positively skewed score distribution, which is not unusual for EM assessment tools [ 10 ]. In that same year, Emery et al evaluated how often professionalism flags were being identified on the NCAT-EM and found that such flags were uncommon but did correlate to overall lower scores when present [ 11 ]. Neither study found significant score differences between genders, however, Hiller et al [ 9 ] did find that fourth-year medical students (MS4s) typically scored higher than third-year medical students (MS3s).…”