Validity Evidence of a Multiple Mini Interview for Selection of Medical Students: Universiti Sains Malaysia Experience

Abstract

Introduction: Student selection is important for selecting the best candidates into medical courses. The Multiple Mini Interview (MMI) is a valid selection tool for the task. The School of Medical Sciences (SMS), Universiti Sains Malaysia (USM) has employed MMI as a medical admission tool to select potential candidates into its medical program. Objective: This paper described the implementation of the MMI and reported the preliminary evaluation data on its validity evidence. Method: A 9-station MMI (with 5 manned and 4 rest stations) was employed for the 2015 student selection exercise. Interview data were analysed and questionnaire surveys were administered to both interviewers and candidates. Validity, reliability, feasibility and acceptability were determined. Results: Unidimensional construct on confirmatory factor analysis (CFA) and interviewer comments provided evidence of construct validity. Overall reliability was 0.94, good enough for high-stakes decisions. Interviewers and candidates' comments on feasibility and acceptability were also generally positive. The consistency of the difficulty and discrimination indices of similar stations between sessions was identified as among the areas for improvement. Conclusions: We are cautiously optimistic regarding the utility of the MMI in the SMS; future improvements are planned but the present implementation seems sufficient for the stated purposes.

Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review

“…Table 1 shows 49 articles reported evidence to support the content of MMI, while Table 2 shows that 40 articles support the internal structure2, 4, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47; 37 articles support the response process2, 5, 6, 11, 12, 14, 18, 19, 20, 23, 25, 26, 27, 28, 30, 31, 33, 37, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62; 21 articles support the relation to other variables,4, 15, 16, 17, 21, 24, 25, 29, 38, 39, 40, 44,…”

“…Internal structure . Six studies reported an MMI internal consistency level (Cronbach's alpha) of less than 0.7,19, 22, 23, 26, 28, 38 whereas 18 articles reported at least 0.70 11, 12, 13, 16, 18, 19, 21, 22, 23, 24, 26, 31, 32, 34, 37, 43, 44, 47. The lowest and highest levels of internal consistency were 0.54 38 and 0.98, 13 respectively (Table 2).…”

“…Response process . Both applicants and examiners were positive about the experience and potential of MMI as a student selection method2, 6, 14, 26, 27, 28, 33, 37, 46, 47, 48, 49, 51, 52, 53, 55, 56, 60, 61; MMI is free of gender, age, previous experience, prior knowledge, and cultural bias14, 19, 23, 31, 48, 54, 57, 60; MMI is a fair assessment and scoring sheet which allowed them to differentiate between applicants5, 18, 19, 20, 47, 48, 51, 52, 58, 60; neither aboriginal-specific rater training nor aboriginal rater assignment is required 11 ; violations of MMI security do not unduly influence applicant performance ratings 12 ; MMI provides sufficient time for students to present ideas 48 ; MMI is at least as cost-efficient as many other personal interview formats, 50 MMI eases interviewer anxiety associated with having to judge candidates unfavourably 51 ; and MMI was not stressful 27 . Conversely, MMI requires a greater number of rooms 50 ; station scores provided by student interviewers were slightly higher than those of faculty member or practitioner interviewers 25 ; student interviewers were less lenient26, 30 and had more unexpected ratings 30 ; students preferred a mixed format, rather than MMI alone 59 ; cultural specificity of some stations and English-language proficiency were seen to disadvantage international students 37 ; applicants with introverted personalities may fare less well in the MMI process 62 ; and raters were unable to distinguish between the various non-cognitive attributes 45 .…”

A comparative study on the psychological health status of pre-clinical medical students enrolled through different admission tests

“…The candidates were assessed on language proficiency, general conduct, critical thinking, ethical awareness, communication skills, knowledge of the healthcare system, and standard interview questions. 20 Ranked on their scores, the top 150 candidates were recommended for the medical program.…”

“…24 Each statement was rated using a 4-point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much). Based on the DASS manual for student samples 22 , 23 , 24 ; 1) stress level is categorized as normal (0–14), mild, 15 , 16 , 17 , 18 moderate, 19 , 20 , 21 , 22 , 23 , 24 , 25 severe, 16 , 26 , 27 , 28 , 29 , 30 , 31 , 32 and extremely severe (34 and above); 2) Anxiety level is categorized as normal (0–7), mild, 8 , 9 moderate, 10 , 11 , 12 , 13 , 14 severe 15 , 16 , 17 , 18 , 19 and extremely severe (20 and above); and 3) Depression level is categorized as normal (0–9), mild, 10 , 11 , 12 , 13 moderate, 14 , 15 , 16 , 17 , 18 , 19 , 20 severe 21 , 22 , 23 , 24 , 25 , 26 , 27 and extremely severe (28 and above). Any scores of moderate to extremely severe levels were considered as unfavourable stress, anxiety, and depression.…”