Key Word Use in Letters of Recommendation for Ophthalmology Residency Applicants According to Race, Gender, and Achievements

Abstract

Objectives To identify differences in letters of recommendation (LORs) of applicants to a single ophthalmology residency program by gender, race, academic performance, and match outcome. Design This was a retrospective analysis of LORS for 2,523 applicants (7,569 letters) to the University of California, Irvine ophthalmology residency program from 2011 to 2018. Methods Programming scripts were employed to determine the number of times 22 key words from four thematic categories (standout words, ability, grindstone, and compassion) appeared in LORs for each applicant. A chi-square test was performed to assess for possible differences in the presence of each key word by the following characteristics: gender, underrepresented minority (URM) status, Alpha Omega Alpha (AOA) membership, the United States Medical Licensing Exam (USMLE) Step 1 score, and match outcome. Linear regressions were created to determine the frequency at which words in each thematic category appeared according to the same baseline characteristics. Results In the LORs, females were more likely to be described as “empathetic” (p = 0.002), URMs were more likely to be described as “caring” (p = 0.002), high Step 1 scorers (≥240) were more likely to be described as “outstanding” (p = 0.002), and matched students were more likely to be described as “exceptional” (p = 0.001), “outstanding” (p < 0.001), and “superb” (p = 0.001). Standout words appeared more often in the LORs of AOA members, matched candidates, and high Step 1 scorers (p < 0.001 for all comparisons). “Competent” appeared more commonly in LORs for low Step 1 scorers (p < 0.001) and unmatched applicants (p = 0.001). Conclusion This study identifies differences in LORs by gender, URM status, and achievement including successful ophthalmology residency match. Females and URMs were more likely to be described as “empathetic” and “caring,” respectively; otherwise, we detected no gender or racial disparities in key word use in LORs. Candidates with high USMLE Step 1 scores or AOA membership had a higher frequency of standout words in their LORs. Whether they were truly more qualified in various dimensions or if they benefited from a halo effect bias warrants further investigation. There was a significant difference in the number of standout words in LORs between matched and unmatched applicants, suggesting that key word frequency may be a relevant metric for LOR appraisal.

Sex Disparities in Ophthalmology From Training Through Practice

“…Most studies (57 [62.6%]) were published in or after 2018 (eTable 1 in the Supplement). Eighty-seven studies [10][11][12][13][14][15][16][18][19][20][21][22][23][24][25][26]99,100 were crosssectional, and 4 studies 17,27,69,98 were cohort. In the risk of bias assessment, 12 had low risk of bias 17,27,30,44,45,69,74,86,[90][91][92]98 (good), 68 were moderate 12-16,18-26,28,29,31-43,47-52,54,55,57-63,65,67,68,70,72,73,75-79,81, 83-85, 87-89, 93-97, 99 (fair), and 11 were high 10,11,46,53,56,64,66,71,80,82,100 (poor) (eTables 2 and 3 in the Supplement).…”

“…Of 4077 retrieved studies, we reviewed 154 full-text articles, with 91 eligible studies published between 1995 and 2022 (Figure). Most studies (57 [62.6%]) were published in or after 2018 (eTable 1 in the Supplement).…”

“…12 Letters for males included more authentic words (eg, written with more honest, personal, and disclosing terms; MD, 0.800; 95% CI, 0-1.60; P = .047) and leisure words (eg, cook, chat, movie; MD, 0.06; 95% CI, 0.01-0.10; P = .02). 12 Another single-center analysis 13 of reference letters for 2523 applicants between 2011 and 2018 reported that the relevance was negligible for sex-based terms (such as empathetic) among matched vs unmatched applicants.…”

“…17 Reviewed logbooks of 133 vitreoretinal fellows between 2018 and 2019 found no sex differences in total procedure volumes at the end of fellowship (median [IQR], 1005 [719-1257] vs 1171 [864-1600]; P = .07). 18 However, for specific procedures, females had lower volumes for internal limiting membrane peel (median [IQR], 29 vs 45 ; P = .02), scleral buckle (median [IQR], 22 vs 35 ; P = .03), and cryoretinopexy (median [IQR], 11 [6][7][8][9][10][11][12][13][14][15] vs 16 [8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]; P = .01). 18…”

Race and Gender Bias in Internal Medicine Program Director Letters of Recommendation

“…Men and White applicants are more likely to be described as ''exceptional'' or ''leaders,'' Black applicants as ''competent,'' and women as ''empathetic'' or ''compassionate.'' [9][10][11] These patterns persist when applicants apply to fellowship. 12 Bias can manifest in many ways in LOR.…”