A Novel Approach to Assessing Professionalism in Preclinical Medical Students Using Multisource Feedback Through Paired Self- and Peer Evaluations

Abstract

Spearman correlation demonstrated moderate to strong correlation between self-assessments completed alone (no simultaneous peer assessment) and self-assessments completed at the time of peer assessments (ρ = .59, p < .0001) but weak correlation between the two self-assessments and peer assessments (alone: ρ = .13, p < .013; at time of peer: ρ = .21, p < .0001). Generalized estimating equation models revealed that self-assessments done alone (p < .0001) were a significant predictor of self-assessments done at the time of peer. Course was also a significant predictor (p = .01) of self-assessment scores done at the time of peer. Peer assessment score was not a significant predictor. Bhapkar's test revealed subgroup membership based on the relationship between self- and peer ratings was relatively stable across Time 1 and Time 2 assessments (χ = 0.83, p = .84) for all but one subgroup; members of the subgroup with initially high self-assessment and low peer assessment were significantly more likely to move to a new classification at the second measurement. A missing data analysis revealed that students who completed all self-assessments had significantly higher average peer assessment ratings compared to students who completed one or no self-assessments with a difference of -0.32, 95% confidence interval [-0.48, -0.15]. Insights: Multiple measurements of simultaneous self- and peer assessment identified a subgroup of students who consistently rated themselves higher on professionalism attributes relative to the low ratings given by their peers. This subgroup of preclinical students, along with those who elected to not complete self-assessments, may be at risk for professionalism concerns. Use of this multisource feedback tool to measure perceptual stability of professionalism behaviors is a new approach that may assist with early identification of at-risk students during preclinical years.

Assessing Professionalism in Medicine – A Scoping Review of Assessment Tools from 1990 to 2018

“…• Undergraduate medical school education 18,20,[24][25][26]28,74,80,83,85,96,98,104,106,107,109,111,115,117,119,120,122,[123][124][125][126][127] • graduate medical school education 88 • Postgraduate clinical training 4,[21][22][23]29,30,[76][77][78][79]81,86,90,[93][94][95]97,99,100,102,103,105,108,110,…”

“…• Likert Scale Questionnaire [22][23][24][28][29][30][76][77][78]83,[85][86][87][88]90,[92][93][94]96,97,99,[101][102][103][104][105][106][107]109,110,111,112,[116][117][118][119][120][121][122][124][125][126][127]129 • Frequency of behaviour scale 21,84 • Yes/No questionnaire 76,84 • Checklist 84,115,128 • Multiple-choice questions/responses…”

“…• Multi-modal 75,118,122 • Multi-source 3,4,23,26,30,74,[80][81][82][83][84]105,108,115,116,122,130 • Self-assessment 20,21,24,25,28,[76][77][78][79]83,85,[88][89][90][97][98][99][100]106,109,[111][112][113][114][115][116][117][119][120][121]126,127,131 ...…”

Peer Feedback on Collaborative Learning Activities in Veterinary Education

“…Several recent reports in the medical education literature have provided evidence that peer feedback outcomes may assist in the identification of students at risk of later professionalism concerns during senior clinical training [60,61,62]. Two retrospective studies [61,62] demonstrated an association between poor peer feedback on collaborative tasks in pre-clinical settings and later professionalism lapses during clinical training.…”

Evaluating the effectiveness of undergraduate clinical education programs

“…For example, assessment of student performance during clerkships should not only include evaluations from faculty and residents, but also from patients, clinical staff, administrative staff, peer students, and even self-evaluations. Multisource Feedback (MSF) approaches, such as 360-degree evaluations, are already used in many residency training programs [31][32][33][34][35][36] and even in some undergraduate medical education programs [37,38]. MSF evaluations can provide valuable insight into the learning environment, increase stakeholder representation in the medical education program, and identify gaps in skill development that may go unrecognized in traditional evaluations.…”