Validity evidence for the Hamburg multiple mini-interview

Knorr, Mirjana; Schwibbe, Anja; Ehrhardt, Maren; Lackamp, Janina; Zimmermann, Stefan; Hampe, Wolfgang

doi:10.1186/s12909-018-1208-0

Cited by 30 publications

(32 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…[6][7][8][9] Knorr et al (2018) examined the suitability of medical students for the medical profession in order to validate their selection procedure based on multiple mini-interviews (MMIs). 10 In their study, supervising GPs were asked to rate medical students after a one-week internship in their practices, following the first year of medical education. The MMI score showed a significant correlation with the suitability (assessed by the same question we used) rating.…”

Section: Discussionmentioning

confidence: 99%

<p>Do Medical Students in Their Fifth Year of Undergraduate Training Differ in Their Suitability to Become a “Good Doctor” Depending on Their Admission Criteria? A Pilot Study</p>

Kötter¹,

Rose²,

Waldmann

et al. 2020

AMEP

View full text Add to dashboard Cite

Background: In Germany, the pre-university grade point average (pu-GPA) has to be the main criterion for medical school applicant selection. This is also mandatory in the university-specific selection procedures (Auswahlverfahren der Hochschulen [AdH]). The admission framework has now been reworked following a judgement by the German Federal Constitutional Court. From 2020, more students will be admitted based solely on the pu-GPA and at least two selection criteria independent of the pu-GPA have to be considered in the AdH. However, the question whether an AdH (the core of the AdH at Lübeck Medical School [LMS], Germany, is a 30-mins panel interview led by two faculty members and one student) leads to better doctors as compared to pu-GPA-based selection, remains unanswered. Objective: To compare students selected based either on their pu-GPA alone ("pu-GPAstudents") or based on the result of the AdH at LMS ("AdH-students") regarding their suitability to become a good doctor. Design: We conducted a cross-sectional observational pilot study at LMS. Students were judged regarding their overall suitability to become a good doctor by their supervising general practitioners after a two-week internship in their last year of theoretical medical education. The scores were matched to the selection procedure and compared between the pu-GPA-students and AdH-students. Results: In all, 79% of the AdH-students were rated as "absolutely suitable" for the medical profession, as compared to 42% of the pu-GPA-students (p = 0.01, odds ratio 5.17, 95% confidence interval = 1.41, 18.99). We did not find any association between gender or age and the suitability rating. Conclusion: Despite the small sample size, our results indicate that it could be favourable to select medical students not only based on their pu-GPA but also using additional selection criteria.

show abstract

Section: Discussionmentioning

confidence: 99%

<p>Do Medical Students in Their Fifth Year of Undergraduate Training Differ in Their Suitability to Become a “Good Doctor” Depending on Their Admission Criteria? A Pilot Study</p>

Kötter¹,

Rose²,

Waldmann

et al. 2020

AMEP

View full text Add to dashboard Cite

show abstract

“…All applicants were administered to MMIs and other selection procedures. The development of the MMIs was very similar to approaches employed at other universities (e.g., Ziv et al 2008;Oliver et al 2014;Knorr et al 2018) and included role-play stations with professional actors and interview stations.…”

Section: Present Researchmentioning

confidence: 99%

“…There is some evidence for the predictive validity of MMIs judgments, (Eva et al 2009; Knorr et al 2018) but what exactly MMIs measure has yet to be concretely identified (Knorr and Hissbach 2014). Specifically, there has been a debate about whether MMIs measure one construct (i.e., a general factor: some applicants perform better than others, regardless of dimensions or stations), one construct per station (i.e., exercise approach: some applicants perform better at some stations than others, regardless of dimensions), or one construct per social skill dimension (i.e., dimensional approach: some applicants perform better on some dimensions than others, regardless of stations).…”

Section: Introductionmentioning

confidence: 99%

Construct Validity of Multiple Mini Interviews – Investigating the Role of Stations, Skills, and Raters using Bayesian G-theory

Breil¹,

Forthmann²,

Hertel³

et al. 2019

Preprint

View full text Add to dashboard Cite

One popular procedure in the medical student selection process are multiple mini-interviews (MMIs), which are designed to assess social skills (e.g., empathy) by means of brief interview and role-play stations. However, it remains unclear whether MMIs reliably measure desired social skills or rather general performance differences that do not depend on specific social skills. Here, we provide a detailed investigation into the construct validity of MMIs, including the identification and quantification of performance facets (social skill-specific performance, station-specific performance, general performance) and their relations with other selection measures. We used data from three MMI samples (N = 376 applicants, 144 raters) that included six interview and role-play stations and multiple assessed social skills. Bayesian generalizability analyses show that, the largest amount of reliable MMI variance was accounted for by station-specific and general performance differences between applicants. Furthermore, there were low or no correlations with other selection measures. Our findings suggest that MMI ratings are less social skill-specific than originally conceptualized and are due more to general performance differences (across and within-stations). Future research should focus on the development of skill-specific MMI stations and on behavioral analyses on the extents to which performance differences are based on desirable skills versus undesired aspects.

show abstract

“…Apart from school leaving grades the individual medical school quota must consider an SAT and an additional criterion independent of school grades. Complex interview procedures are possible and grounded on evidence [6], [7], [17]. Only in case of the use of complex admission procedures the individual medical school may limit the admission to applicants choosing the respective medical school as first priority.…”

Section: Editorialmentioning

confidence: 99%