Intern preparedness for the CanMEDS roles and the Dunning-Kruger effect: a survey

Abstract

Background The purpose of this study was to determine whether the first cohort of graduates from a new undergraduate medical programme in Botswana were adequately prepared for internship. Methods The authors surveyed 27 interns and 13 intern supervisors on site, who rated intern preparedness for 44 tasks using a previously validated instrument. Tasks were grouped according to the seven roles of the physician in the CanMEDS framework and Cronbach α values confirmed internal consistency. To determine the direction of differences between intern and supervisor ratings for tasks Likert scale ratings were treated as interval data and average scores calculated. Rating frequencies for each role were compared using the χ2 statistic. Reasons for differences between intern and supervisor ratings were explored by determining correlations between scores using the Pearson r statistic. Results Preparedness for all seven roles and the majority of tasks was found to be between ‘Fairly well prepared’ and ‘Well prepared’. The ratings for four roles (Medical expert, Communicator, Collaborator, Professional) differed statistically, but not for the three others (Leader, Health advocate, Scholar). Interns rated their proficiency higher than their supervisors for the tasks in six roles; for the ‘Professional’ role intern ratings were mostly lower. Correlations between intern and supervisors scores were only statistically significant for three roles (Medical expert, Communicator, Collaborator). Conclusions Intern preparedness for tasks and roles varied but was generally satisfactory. Based on the analysis of the data seeming discrepancies in between interns and supervisor ratings were investigated and explanations are offered. For three roles the data indicate that their component tasks are understood in the same way by interns and supervisors, but not for the other roles. The Dunning-Kruger effect offers a plausible explanation for higher intern scores for tasks in six of the roles. For the ‘Professional’ role differences between interns’ internal, individual understanding and supervisors’ external, group understanding may explain lower intern scores. The fact that respondents may understand the tasks they rate differently has implications for all research of this nature.

Transitioning from university to postgraduate medical training: A narrative review of work readiness of medical graduates

“…There were a further 10 studies including literature reviews, commentaries, and editorials (Table 1). Study participants included final‐year medical students, 6‐25 junior doctors early in their first postgraduate year, 3,13,14,16‐19,26‐42 junior doctors late in internship (>6 months) or in postgraduate year 2 or above 2,14,16,22,24,35,38,42‐57 (Table 1). Twenty‐one studies incorporated feedback or reports from clinical supervisors or training managers 5,7,13,14,17‐19,22,25,31,35,36,39,43,45,55,58‐61 (Table 1).…”

“…An audit of workplace‐based assessments in South Australia demonstrated that supervisors rated interns at or above “competent/well prepared,” whereas graduates rated themselves more harshly 45 . In contrast, supervisor ratings of intern preparedness were lower compared to interns' self‐ratings in Scotland 43 and Botswana 42 . Several sources reported a lack of graduate preparedness for prescribing 8,15,18 ; however, only one study objectively measured prescribing competencies using a “safe prescribing” assessment 18 .…”

“…There were a further 10 studies including literature reviews, commentaries, and editorials (Table 1). Study participants included final-year medical T A B L E 1 Study design and participants in papers included in the review of literature 1998 -2019 Dyar 12 McKenzie 2017 15 Minha 16 Morrow 17 Rothwell 18 Scicluna 19 Stroben 21 Wijnen-Meijer 2015 23 Daly 25 Illing 2013 14 Illing Lightmann Tran 24 Watmough 22 Early internship Abuhusain 26 Bennett 27 Bleakley 28 Cave 29 Heidemann 31 Minha 16 Morrow 17 Prozesky 42 Scicluna 19 Watmough 22 Winn 40 Eley 2 Illing 2013 14 Lundin 32 Monrouxe 2018 35 Muthaura 36 Sturman 3 Walker 39 Gome 30 Illing Kelly 37 Linane 38 McKenzie 41 Miles 33 Minter 34 Rothwell 18 Late or post internship Barr 53 Dean 55 Duns 52 Eley 62 Goldacre 2010 50 Goldacre 2014 57 Hill 56 Kassim 49…”

“…In around onethird of studies, other stakeholders' views were incorporated through interviews with clinical supervisors, 13,14,18,25,35,36,61 training managers 18,35,39,55 or other staff (eg, nurses, pharmacists), 13,18,35 or from supervisor surveys or reports. 17,31,42,43,45,[58][59][60] One study examined a stand-alone assessment of work readiness (a multifaceted simulation scenario), 23 and another study analyzed results of a safe prescribing assessment. 18 Only two studies explored patient or carer perspectives of junior doctors.…”

“…45 In contrast, supervisor ratings of intern preparedness were lower compared to interns' self-ratings in Scotland 43 and Botswana. 42 Several sources reported a lack of graduate preparedness for prescribing 8,15,18 ; however, only one study objectively measured prescribing competencies using a "safe prescribing" assessment. 18 Clinicians and patients perceived that junior doctors generally did not manage more complex communication tasks well, such as considering psychosocial aspects of patient care or involving patients in decision-making.…”