The efficacy of medical student selection tools in Australia and New Zealand

Abstract

The known The predictive power of composite scores based on the weighted means of the results from multiple selection tools is low, explaining less than 15% of variance in student outcomes, regardless of the statistical models, selection tools, and outcomes used.The new Discriminant function analysis yields optimal and meaningful cut-scores for each of the selection tools based on binary outcomes. A non-compensatory selection model or a "sufficient evidence" approach to selection would be useful for some medical schools.The implications These alternative approaches may enhance the efficacy and defensibility of the selection process. They may also minimise the student non-completion rate in medical programs.

Selecting top candidates for medical school selection interviews- a non-compensatory approach

“…For example at the University of New South Wales, Sydney, Australia (UNSW) only about one in ten applicants succeed in securing a place in the Medicine program. Medical schools apply a range of selection methods to ensure that admitted students succeed in the program, complete the their studies in a timely manner and become competent junior doctors [2]. The most common selection tools used by Australian undergraduate medical schools is the Australian Tertiary Admission Rank (ATAR) or equivalent, which is a score between 0.00 and 99.95 based on high school matriculation examinations, indicating a student's position in their cohort [3].…”

“…The UMAT was devised by the Australian Council for Educational Research, and consists of three sections: logical reasoning and problem solving; understanding people; and non-verbal reasoning. Most undergraduate medical programs in Australia use ATAR and UMAT outcomes to select a smaller group of applicants for additional selection processes, which may include interviews, psychological tests, or both [2,[6][7][8][9]. Most of the literature focusing on the efficacy of selection to medical programs focuses on the final stage of the selection process; that is selection from the pool of applicants who were successful in being offered an interview [2,[9][10][11].…”

“…Most undergraduate medical programs in Australia use ATAR and UMAT outcomes to select a smaller group of applicants for additional selection processes, which may include interviews, psychological tests, or both [2,[6][7][8][9]. Most of the literature focusing on the efficacy of selection to medical programs focuses on the final stage of the selection process; that is selection from the pool of applicants who were successful in being offered an interview [2,[9][10][11]. A few medical schools do not require an interview and use only academic and cognitive test achievements to select medical students [2,10].…”

A Signal Detection Approach in a Multiple Cohort Study: Different Admission Tools Uniquely Select Different Successful Students

“…From a societal viewpoint, the admission procedure resulting in the lowest proportion of rejected potentially successful students would contribute best to the development and use of human potential. To date, we have not encountered other methods that provide information on the proportions of both types of mistakes resulting from different tools and criteria, though methods complementary to regressions have been reported (e.g., Shulruf et al, 2018). In medicine, Signal Detection Theory is commonly used in investigating the accuracy of diagnostic tests.…”

Selecting medical students: we need to assess more than academic excellence

“…In this issue of the MJA , Shulruf and his co‐authors 5 argue that completion of medical school is a better proxy measure of competence than exam performance per se because it better reflects real world outcomes and therefore, presumably, future patient safety. The authors analysed this outcome at five Australian and New Zealand universities, and found that prior educational attainment was the best predictor, aptitude testing (the Undergraduate Medicine and Health Sciences Admission Test [UMAT]) and interview not markedly improving discrimination between students who succeeded or failed.…”