Reliability and acceptability of a five-station multiple mini-interview model for residency program recruitment

Abstract

BackgroundStandard interviews are used by most residency programs in the United States for assessment of aptitude of the non-cognitive competencies, but variability of interviewer skill, interviewer bias, interviewer leniency or stringency, and context specificity limit reliability.AimTo investigate reliability and acceptability of five-station multiple mini-interview (MMI) model for resident selection into an internal medicine residency program in the United States.SettingOne independent academic medical center.ParticipantsTwo hundred and thirty-seven applicants and 17 faculty interviewers.Program descriptionFive, 10-min MMI stations with five different interviewers blinded to the candidate's records and one traditional 20-min interview with the program director. Candidates were rated on two items: interpersonal and communication skills, and overall performance.Program evaluationGeneralizability data showed that the reliability of our process was high (>0.9). The results of anonymous surveys demonstrated that both applicants and interviewers consider the MMI as a fair and more effective tool to evaluate non-cognitive traits, and prefer the MMI to standard interviews.DiscussionThe MMI process for residency interviews can generate reliable interview results using only five stations, and it is acceptable and preferred over standard interview modalities by the applicants and faculty members of one US residency program.

Validity Evidence of a Multiple Mini Interview for Selection of Medical Students: Universiti Sains Malaysia Experience

“…The reliability of 0.94 in our exercise is understandable as in our MMI, 447 candidates were rated using six identical circuits over several sessions. It is reassuring to note that a similar 5-station MMI which rated 237 candidates also reported a similar reliability of > 0.9 (33). Despite the reassuring data, we will still consider adding more stations to future MMIs as long as feasibility permits, as common assessment sense would suggest that it will increase the validity and reliability further.…”

Rater Perceptions of Bias Using the Multiple Mini-interview Format: a Qualitative Study

“…The MMI was found to be reliable (r=0.7-0.85) using individual interviewers in 6-12 stations that ranged from 5.5-10 minutes each (Eva et al, 2004;Harris & Owen, 2007;Hecker et al, 2009;Hofmeister, Lockyer, & Crutcher, 2009;Roberts et al, 2008). In addition, the MMI demonstrated predictive validity for future assessments of non-cognitive performance (Eva et al, 2009;Lemay, Lockyer, Collin, & Brownell, 2007;Reiter, Eva, Rosenfeld, & Norman, 2007;Roberts et al, 2008), was acceptable to applicants and interviewers (Diaz Fraga, Oluwasanjo, Wasser, Donato, & Alweis, 2013;Humphrey, Dowson, Wall, Diwakar, & Goodyear, 2008;Kumar, Roberts, Rothnie, du Fresne, & Walton, 2009;Razack et al, 2009) and cost-effective (Rosenfeld, Reiter, Trinh, & Eva, 2008), leading to its widespread adoption as an interviewing format in Canada, the United States, and Australia (Eva et al, 2004;B. Griffin & Wilson, 2012;Kreiter, Yin, Solow, & Brennan, 2004).…”

Mini Surgical Simulation, Role Play, and Group and Behavioral Interviews in Resident Selection

“…Inherent problems of traditional interviews are the variability in the interviewers' communication skills, bias, tolerance, and personality. [1][2][3][4][5] Many studies have shown that traditional interviews have poor interrater reliability and perceived gender and cultural bias, with inadequate reliability and questionable validity. [1][2][3][4][5][6] The multiple mini interview (MMI) is a standardized, validated selection process that consists of several stations, each of which assesses different aspects of the applicants; the MMI is used in medical schools in Canada and has spread to other nations.…”

“…[1][2][3][4][5] Many studies have shown that traditional interviews have poor interrater reliability and perceived gender and cultural bias, with inadequate reliability and questionable validity. [1][2][3][4][5][6] The multiple mini interview (MMI) is a standardized, validated selection process that consists of several stations, each of which assesses different aspects of the applicants; the MMI is used in medical schools in Canada and has spread to other nations. [1][2][3][4][5][6][7][8] The behavioral format utilizes questions that relate to past behaviors and performance as a way to identify latent professionalism characteristics, and this approach is predictive of future behavior.…”

“…[1][2][3][4][5][6] The multiple mini interview (MMI) is a standardized, validated selection process that consists of several stations, each of which assesses different aspects of the applicants; the MMI is used in medical schools in Canada and has spread to other nations. [1][2][3][4][5][6][7][8] The behavioral format utilizes questions that relate to past behaviors and performance as a way to identify latent professionalism characteristics, and this approach is predictive of future behavior. 9 To identify desirable attributes in residency applicants (cultural fit, communication skills, teamwork, trainability, and professionalism), we developed a multifaceted interview process.…”