Qualitative analysis of MMI raters’ scorings of medical school candidates: A matter of taste?

Abstract

Recent years have seen leading medical educationalists repeatedly call for a paradigm shift in the way we view, value and use subjectivity in assessment. The argument is that subjective expert raters generally bring desired quality, not just noise, to performance evaluations. While several reviews document the psychometric qualities of the Multiple Mini-Interview (MMI), we currently lack qualitative studies examining what we can learn from MMI raters' subjectivity. The present qualitative study therefore investigates rater subjectivity or taste in MMI selection interview. Taste (Bourdieu 1984) is a practical sense, which makes it possible at a pre-reflective level to apply 'invisible' or 'tacit' categories of perception for distinguishing between good and bad. The study draws on data from explorative in-depth interviews with 12 purposefully selected MMI raters. We find that MMI raters spontaneously applied subjective criteria-their taste-enabling them to assess the candidates' interpersonal attributes and to predict the candidates' potential. In addition, MMI raters seemed to share a taste for certain qualities in the candidates (e.g. reflectivity, resilience, empathy, contact, alikeness, 'the good colleague'); hence, taste may be the result of an ongoing enculturation in medical education and healthcare systems. This study suggests that taste is an inevitable condition in the assessment of students' performance. The MMI set-up should therefore make room for MMI raters' taste and their connoisseurship, i.e. their ability to taste, to improve the quality of their assessment of medical school candidates.

How organizational culture influences holistic review: a qualitative multiple case study

“…In their eloquent study of Multiple Mini-Interview (MMI) interviewers’ ‘taste’—defined as “…individuals’ subjective judgments as a matter of practical sense”—Christensen and colleagues (2018, p. 292) describe the influence of alters (e.g., role models or leaders) on actors who, in the context of admissions, are applicant raters. They note that according to socialist Crossley ( 2013 ), “Alters teach actors how to appreciate and enjoy cultural objects that they might not otherwise ‘get’” (in Christensen et al, 2018 , p. 291). Consequently, actors develop shared appreciation for alters’ tastes.…”

“…Consequently, actors develop shared appreciation for alters’ tastes. (Christensen et al, 2018 ). Christensen et al conclude that medical school applicant raters’ similar preferences for particular attributes may partially result from “shared habituated norms.” (Christensen et al, 2018 , p. 301).…”

“…(Christensen et al, 2018 ). Christensen et al conclude that medical school applicant raters’ similar preferences for particular attributes may partially result from “shared habituated norms.” (Christensen et al, 2018 , p. 301). They also note that enculturation appears to profoundly influence rater preferences for candidates who have attributes and characteristics with which they can identify.…”

“…By illuminating the influence that social interactions, habituated norms, and values may have on subjective rater judgments, it could be said that Christensen et al ( 2018 ) demonstrated the influence of culture. Although we did not focus on how raters (i.e., faculty) form their impressions, it’s clear their judgments are influenced by program culture.…”

<p>Multiple Mini-Interviews: Current Perspectives on Utility and Limitations</p>

“…MMIs showed discriminant validity in many studies comparing no correlations with past academic performances like grade point average (GPA)33 or logical reasoning ability or non-verbal reasoning 40…”