Lessons learned from 15 years of non-grades-based selection for medical school

Abstract

ContextThirty years ago, it was suggested in the Edinburgh Declaration that medical school applicants should be selected not only on academic, but also on non‐academic, attributes. The main rationale behind extending medical school selection procedures with the evaluation of (non‐academic) personal qualities is that this will lead to the selection of students who will perform better as a doctor than those who are selected on the basis of academic measures only. A second rationale is the expectation that this will lead to a representative health workforce as a result of reduced adverse impact. The aims of this paper are (i) to describe what can be learned about the use of selection criteria other than grades from over 15 years of Dutch experience and (ii) to summarise current knowledge on the issue of adverse impact in relation to non‐grades‐based selection.MethodsA narrative review was undertaken of the (published) evidence that has resulted from non‐grades‐based school‐specific selection procedures in the Netherlands and from recent explorations of the effect of the use of non‐grades‐based selection criteria on student diversity.ResultsThe Dutch evidence is grouped into five key themes: the effect of participation in voluntary selection procedures, the assessment of pre‐university extracurricular activities, the use of work samples, Dutch experiences with situational judgement tests and the effects of changing circumstances. This is followed by several lessons learned for medical schools that aim to increase their student diversity.ConclusionOver the last 30 years, important steps towards reliable and valid methods for measuring non‐academic abilities have been taken. The current paper describes several lessons that can be learned from the steps taken in the Dutch context. The importance of sharing evidence gathered around the globe and building on this evidence to reach our goal of predicting who will be a good doctor is acknowledged.

Selecting medical students: considering qualities other than academic ability

“…Stegers‐Jager's findings, which support those of others, suggest that non‐academic/non‐cognitive qualities appear to be better at predicting students’ performance later rather than earlier in their medical studies, when typically the focus turns more to clinical skill acquisition than to the learning of knowledge. She thus raises the question of whether we should be selecting applicants based on qualities that are required to be a good medical student or those required to be a good doctor . However, are these qualities likely to differ substantially?…”

“…Although we should draw on the large body of research on selection in the corporate world, medical student selection has some unique aspects that point to the need for targeted predictive validity studies, and yet there are comparatively few. Therefore, the national system that existed for selecting students in the Netherlands, as described by Karen Stegers‐Jager in this issue, did indeed provide a novel opportunity to evaluate the predictive capacity of selection systems that include the assessment of personal qualities. Stegers‐Jager presents compelling evidence of the benefits of these systems over a selection system that assesses only academic ability for key outcomes such as dropout rate and clinical performance.…”

“…Like most, Stegers‐Jager focuses almost exclusively on positive traits and skills, such as teamwork skills, empathy and resilience, when discussing important non‐cognitive qualities to be assessed at the point of selection . However, when I ask faculty members to describe the specific qualities they want their medical students to possess, they more often than not first mention the distinctly negative things they don't want, such as ‘a sense of entitlement’ or ‘psychopathic’ traits.…”

Who is allowed to study medicine? – regulations and evidence

“…Nevertheless, student selection internationally includes a broad range of procedures with varying evidence, ranging from selection by lottery [2] to mere selection on the ground of school leaving grades [3]. Additional student admission tests (SAT) are frequently used, since their results reliably predict study success as measured by cognitive outcome criteria [4], [5].…”

A Signal Detection Approach in a Multiple Cohort Study: Different Admission Tools Uniquely Select Different Successful Students

“…Research into predictors other than secondary school grade point average (SSGPA)-which is far from accurate-is still work in progress, and a "gold standard" has not yet been conceived (Richardson et al, 2012;Schripsema et al, 2014Schripsema et al, , 2017Shulruf and Shaw, 2015;Makransky et al, 2016;Patterson et al, 2016;Pau et al, 2016;Sladek et al, 2016;Yhnell et al, 2016;Wouters et al, 2017). Program-and institution specific work samples appear promising valuable predictors of academic achievement in addition to past academic achievement (Niessen et al, 2016;Stegers-Jager, 2017;van Ooijen-van der Linden et al, 2017).…”