Linguistic Differences by Gender in Letters of Recommendation for Minimally Invasive Gynecologic Surgery Fellowship Applicants

Objectives Inter- and intra-observer variability is a concern for medical school admissions. Artificial intelligence (AI) may present an opportunity to apply a fair standard to all applicants systematically and yet maintain sensitivity to nuances that have been a part of traditional screening methods. Material and Methods Data from 5 years of medical school applications were retrospectively accrued and analyzed. The applicants (m = 22 258 applicants) were split 60%–20%–20% into a training set (m = 13 354), validation set (m = 4452), and test set (m = 4452). An AI model was trained and evaluated with the ground truth being whether a given applicant was invited for an interview. In addition, a “real-world” evaluation was conducted simultaneously within an admissions cycle to observe how it would perform if utilized. Results The algorithm had an accuracy of 95% on the training set, 88% on the validation set, and 88% on the test set. The area under the curve of the test set was 0.93. The SHapely Additive exPlanations (SHAP) values demonstrated that the model utilizes features in a concordant manner with current admissions rubrics. By using a combined human and AI evaluation process, the accuracy of the process was demonstrated to be 96% on the “real-world” evaluation with a negative predictive value of 0.97. Discussion and Conclusion These results demonstrate the feasibility of an AI approach applied to medical school admissions screening decision-making. Model explainability and supplemental analyses help ensure that the model makes decisions as intended.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Linguistic Differences by Gender in Letters of Recommendation for Minimally Invasive Gynecologic Surgery Fellowship Applicants

Cited by 5 publications

References 22 publications

Commentary: The importance of equity in letters of recommendation in residency and fellowship applications

Commentary: The importance of equity in letters of recommendation in residency and fellowship applications

Letters of recommendation for gynecologic oncology fellowship differ by gender and race

Using artificial intelligence in medical school admissions screening to decrease inter- and intra-observer variability

Contact Info

Product

Resources

About