Predicting success in medical school: a longitudinal study of common Australian student selection tools

Abstract

BackgroundMedical student selection and assessment share an underlying high stakes context with the need for valid and reliable tools. This study examined the predictive validity of three tools commonly used in Australia: previous academic performance (Grade Point Average (GPA)), cognitive aptitude (a national admissions test), and non-academic qualities of prospective medical students (interview).MethodsA four year retrospective cohort study was conducted at Flinders University Australia involving 382 graduate entry medical students first enrolled between 2006 and 2009. The main outcomes were academic and clinical performance measures and an indicator of unimpeded progress across the four years of the course.ResultsA combination of the selection criteria explained between 7.1 and 29.1 % of variance in performance depending on the outcome measure. Weighted GPA consistently predicted performance across all years of the course. The national admissions test was associated with performance in Years 1 and 2 (pre-clinical) and the interview with performance in Years 3 and 4 (clinical). Those students with higher GPAs were more likely to have unimpeded progress across the entire course (OR = 2.29, 95 % CI 1.57, 3.33).ConclusionsThe continued use of multiple selection criteria to graduate entry medical courses is supported, with GPA remaining the single most consistent predictor of performance across all years of the course. The national admissions test is more valuable in the pre-clinical years, and the interview in the clinical years. Future selections research should develop the fledgling research base regarding the predictive validity of the Graduate Australian Medical School Admissions Test (GAMSAT), the algorithms for how individual tools are combined in selection, and further explore the usefulness of the unimpeded progress index.

The efficacy of medical student selection tools in Australia and New Zealand

“…It has been reported that the predictive power of composite scores based on the weighted means of the results from multiple selection tools is low, explaining less than 15% of variance in student outcomes, regardless of the statistical models, selection tools, and outcomes used 1 . ‐ 4 , 9 ‐ 11 , 13 , 14 , 16 , 23 We found similar effect sizes to previous studies, but the comprehensiveness and novel types of analyses are the major strengths of our study. We analysed large datasets from five medical schools that applied the student selection tools in different ways.…”

“…We analysed large datasets from five medical schools that applied the student selection tools in different ways. As a result, our results may be more generalisable than those of studies that have focused on a single medical school, or on a particular tool in a particular jurisdiction 1 , 8 , 23 . It is the broadest study of its type to date.…”

A Signal Detection Approach in a Multiple Cohort Study: Different Admission Tools Uniquely Select Different Successful Students

“…Psychology students admitted through an admission procedure including a cognitive admission tests and Multiple Mini Interviews have been shown to outperform psychology students admitted based on their SSGPA (Makransky et al, 2016). Although it is also consistent with most other available information, most studies investigating predictors of academic success other than SSGPA focus on medical programs, and those are currently undecided on the "best practices" in the implementation of admission tools other than SSGPA (Richardson et al, 2012;Schripsema et al, 2014Schripsema et al, , 2017Shulruf and Shaw, 2015;Patterson et al, 2016;Pau et al, 2016;Sladek et al, 2016;Wouters et al, 2017). Note that NCS is quite susceptible to social desirability and that the literature does not allow firm conclusions on the validity of these and other non-cognitive factors (Patterson et al, 2016).…”

“…European university programs use program-specific admission procedures and tools, partly depending on how access to university programs is regulated for these programs, or in specific countries (De Witte and Cabus, 2013;Schripsema et al, 2014;Makransky et al, 2016). Research into predictors other than secondary school grade point average (SSGPA)-which is far from accurate-is still work in progress, and a "gold standard" has not yet been conceived (Richardson et al, 2012;Schripsema et al, 2014Schripsema et al, , 2017Shulruf and Shaw, 2015;Makransky et al, 2016;Patterson et al, 2016;Pau et al, 2016;Sladek et al, 2016;Yhnell et al, 2016;Wouters et al, 2017). Program-and institution specific work samples appear promising valuable predictors of academic achievement in addition to past academic achievement (Niessen et al, 2016;Stegers-Jager, 2017;van Ooijen-van der Linden et al, 2017).…”

Can an early selection system accelerate the maturity of candidates of both genders in medical school?

“…Academic success at university depends on many variables, including high school education, social and economic background, cultural origins and behavioural elements 1 , 2 . Factors for medical school success include intrinsic motivation, intelligence quotient, emotional quotient, regularity of work, sense of self-efficacy, maturity and creativity 3 , 4 .…”