Outcomes-Based Selection Into Medical School: Predicting Excellence in Multiple Competencies During the Clinical Years

Abstract

1. Team Biographies Criterion validity: Concurrent A form of criterion-related validity in which data on the predictor and criterion are obtained at the same time. High correlations between predictor and criterion scores indicate concurrent validity. Criterion validity: Predictive This is the extent to which a predictor measure (e.g. a selection test score) is correlated to A criterion measure (e.g. work performance). High predictive validity indicates that a selection measure gives an accurate indication of candidates' future performance on the criterion. Incremental validity This is an empirical issue to determine how much additional value using another test/assessment provides. Construct validity An indication of the extent to which the test or procedure measures the construct that it is intended to measure (such as empathy, clinical expertise). Political Validity An indication of the extent to which various stakeholders and stakeholder groups (such as, employers, parents, government departments, society, the regulator) consider the tool(s) to be appropriate and acceptable for use in selection. This review considers many validity studies. Although we have endeavoured to minimise technical language, it is helpful for the reader to have a framework for interpreting the validity data which we present. The most commonly used measure of predictive validity is a correlation (or validity) coefficient. The validity coefficient is a statistical index used to report evidence of validity for intended interpretations of test scores and defined as the magnitude of the correlation between test scores and a criterion variable (i.e., a measure representing a theoretical component of the intended

The predictive validity of admission criteria for the results of clinical competency assessment with an emphasis on family medicine in the fifth year of medical education: an observational study

“…MMIs are claimed to be a valid, cost-effective alternative to conventional admission interviews and suitable for the assessment of interpersonal skills (i.e., empathy and communication skills). Evidence for their predictive validity regarding interpersonal skills from prospective, longitudinal studies is still scarce [ 29 ], as for traditional selection interviews [ 14 , 15 ].…”

“…Considering the immense effort for university-specific selection procedures and the necessity to select those who are most likely to become ‘good doctors’, the question arises as to whether students selected by this kind of procedure are more likely to become ‘good doctors’, compared to students selected solely based on their pu-GPA. There are a few studies showing that selection procedures also based on desirable competencies rather than solely based on pu-GPA may be efficient in selecting applicants more likely to develop in a favourable way [ 11 – 15 ]. However, most of these studies compare students in different admission quotas only until the third year of undergraduate training.…”

“…However, most of these studies compare students in different admission quotas only until the third year of undergraduate training. Schreurs et al [ 15 ] showed students admitted via a university-specific, outcome-based selection procedure to outperform their initially rejected but lottery-admitted counterparts at the end of their clinical training regarding different CanMEDS roles. Sladek et al [ 14 ] retrospectively matched different selection score results to competence assessments of interns in an Australian health network.…”

BMAT's predictive validity for medical school performance: A retrospective cohort study

“…[1][2][3][4] Medical school selection processes thus aim to identify the 'best' applicants: those who will do well on medical school and other assessments necessary for progression; become competent clinicians; and deliver care that meets the health care needs of diverse populations. [4][5][6] How to define 'best' continues to be actively debated, 7,8 and there is increasing awareness that definitions may differ by context. 4 However, in practical terms, schools across the world are regularly required to select the best candidates from a pool of well-qualified applicants, many of whom also possess the personal qualities considered desirable in a medical student and doctor.…”

The CASPer Preparation Program Innovation: Increasing Self-Perceived Competence and Confidence of Underrepresented Applicants on the Novel CASPer Snapshot and CanMEDS Roles

“…Some medical schools also require situational judgement tests (SJTs) such as the Computer-based Assessment for Sampling Personal Characteristics (CASPer). [7][8][9][10] A competency-based framework approach called the Canadian Medical Education Directives for Specialists (CanMEDS) includes six key roles de ning the Medical Expert: Communicator, Collaborator, Health Advocate, Manager, Scholar and Professional 11,12 . Admission o ces in Canadian medical schools, such as the Temerty Faculty of Medicine at the University of Toronto, are also increasingly requiring their matriculants to embody the CanMEDS competencies.…”