Evaluation of Validity Evidence for Personality, Emotional Intelligence, and Situational Judgment Tests to Identify Successful Residents

Abstract

This study found little support for the use of EQ assessment and only weak support for some distinct personality factors (ie, agreeableness, extraversion, and independence) in surgery resident selection. Performance on the SJT was associated with overall resident performance more than traditional cognitive measures (ie, USMLE scores). These data support further exploration of these 2 screening assessments on a larger scale across specialties and institutions.

Situational judgement test validity for selection: A systematic review and meta‐analysis

“…A total of 10 were cross-sectional studies, [27][28][29][30][31][32][33][34][35]55 where the outcome was measured at the same time or in the same selection cycle as taking the SJT. A total of 17 were cohort studies 17,[36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51] that had a follow-up period before the outcome of interest was measured. Three studies employed a mixture of F I G U R E 1 PRISMA (preferred reporting items for systematic reviews and meta-analyses) flowchart for the systematic review Abbreviations: CINAHL, Cumulative Index to Nursing and Allied Health Literature; EMBASE, Excerpta Medica Database; ERIC, Educational Resources Information Center; EThos, Electronic Theses Online Service; UCAT, University Clinical Aptitude Test cross-sectional and more distal outcomes.…”

“…[52][53][54] The length of followup across the cohort studies varied from 1 to 9 years after taking the SJT. Full details of the included studies are listed in Table S1. A total of 11 studies 17,28,29,35,[40][41][42][43]47,49,50 looked at undergraduate selection for medical school entry, five studies 31,36,37,45,51 at entry to Foundation Year training programmes (the first 2 years of post-qualification training in the United Kingdom [UK]) and 14 studies 27,30,[32][33][34]38,39,44,46,48,[52][53][54][55] at entry to specialty training. The youngest participant mean age was 17.9 years and the oldest was 34.0 years.…”

“…Most studies that gave details on sample demographics reported having a majority of female participants. The studies were all conducted in high-income countries, with 19 [27][28][29][30][31][32][33]36,37,[44][45][46][47][48][49][51][52][53]55 assessing UK populations, six 17,[40][41][42][43]50 from mainland Europe, three 35,38,39 from North America, and two 34,54 from Australia.…”

Making progress on identifying those who aren’t making progress: Using situational judgment tests to predict those at risk for remediation and attrition

“…For example, the program involved in this study could administer their customized SJT to new interns to identify individuals who are more likely to struggle during clinical training and develop educational interventions accordingly. Additionally, these data contribute to the increasingly robust chain of validation reasoning that suggests SJTs may play an important role in postgraduate medical education selection (Gardner and Dunkin, 2018a;Gardner and Dunkin, 2018b;Patterson et al 2017).…”

“…Semi-structured interviews with subject matter experts (SMEs) and review of historical resident performance data were used to identify which competencies were critical for residents to be successful in the program. Data from these interviews informed the development of SJT items, similar to prior studies (Gardner and Dunkin 2018a;Lievens, Peeters, and Schollaert, 2008). Faculty SMEs central to the residency education program (Program Director, Associate Program Directors, members of the Clinical Competency Committee, etc.)…”