AI and Holistic Review

Alvero, AJ; Arthurs, Noah; Antonio, Anthony Lising; Domingue, Benjamin W.; Gebre-Medhin, Ben; Giebel, Sonia; Stevens, Mitchell L.

doi:10.1145/3375627.3375871

Cited by 24 publications

(12 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fairness is an old and fundamental concept when dealing with data that should cover all data processing activities, from data gathering to data cleansing, through modelling and model deployment. It is not simply that data are biased and this can be amplified by algorithms, but rather that data processing can introduce more bias, from data collection procedures to model deployment 10–17 …”

Section: Introductionmentioning

confidence: 99%

Missing the missing values: The ugly duckling of fairness in machine learning

et al. 2021

View full text Add to dashboard Cite

Nowadays, there is an increasing concern in machine learning about the causes underlying unfair decision making, that is, algorithmic decisions discriminating some groups over others, especially with groups that are defined over protected attributes, such as gender, race and nationality. Missing values are one frequent manifestation of all these latent causes: protected groups are more reluctant to give information that could be used against them, sensitive information for some groups can be erased by human operators, or data acquisition may simply be less complete and systematic for minority groups. However, most recent techniques, libraries and experimental results dealing with fairness in machine learning have simply ignored missing data. In this paper, we present the first comprehensive analysis of the relation between missing values and algorithmic fairness for machine learning: (1) we analyse the sources of missing data and bias, mapping the common causes, (2) we find that rows containing missing values are usually fairer than the rest, which should discourage the consideration of missing values as the uncomfortable ugly data that different techniques and libraries for handling algorithmic bias get rid of at the first occasion, (3) we studyThis is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Section: Introductionmentioning

confidence: 99%

Missing the missing values: The ugly duckling of fairness in machine learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…From the applicants' perspective, Gupta et al and Kiaghadi et al used machine learning techniques to evaluate applicants' chances of admission to help students make informed decisions about where to apply to college [14,17]. In the context of college essays, Alvero et al explored the use of computational text analysis to assist human readers in their evaluation of college application essays [1,2].…”

Section: Related Workmentioning

confidence: 99%

“…The Common Application is the primary tool that applicants use to apply to colleges in the United States. 1 It received over 6.6 million first-year applications during the 2021-22 admission cycle, which constitutes a 9.1% increase over the previous year, and a 21.3% increase over the 2019-20 cycle [13]. The growing number of applications received by selective colleges and universities has presented an annual challenge for college admission, especially for institutions that follow a holistic review process.…”

Section: Introductionmentioning

confidence: 99%

Evaluating a Learned Admission-Prediction Model as a Replacement for Standardized Tests in College Admissions

Lee¹,

Joachims²,

Kizilcec³

2023

Preprint

View full text Add to dashboard Cite

A growing number of college applications has presented an annual challenge for college admissions in the United States. Admission offices have historically relied on standardized test scores to organize large applicant pools into viable subsets for review. However, this approach may be subject to bias in test scores and selection bias in test-taking with recent trends toward test-optional admission.We explore a machine learning-based approach to replace the role of standardized tests in subset generation while taking into account a wide range of factors extracted from student applications to support a more holistic review. We evaluate the approach on data from an undergraduate admission office at a selective US institution (13,248 applications). We find that a prediction model trained on past admission data outperforms an SAT-based heuristic and matches the demographic composition of the last admitted class. We discuss the risks and opportunities for how such a learned model could be leveraged to support human decision-making in college admissions.

show abstract

“…18 Universities in the United States, some of which receive more than 100,000 applications per year, have been using AI and expert systems to augment the screening and review process. 19,20 However, little has been described in the literature on using such approaches to address the increasing volume of medical school applications.…”

mentioning

confidence: 99%

Artificial Intelligence Screening of Medical School Applications: Development and Validation of a Machine-Learning Algorithm

Triola¹,

Reinstein²,

Marin³

et al. 2023

Academic Medicine

View full text Add to dashboard Cite

PurposeTo explore whether a machine-learning algorithm could accurately perform the initial screening of medical school applications. MethodUsing application data and faculty screening outcomes from the 2013 to 2017 application cycles (n = 14,555 applications), the authors created a virtual faculty screener algorithm. A retrospective validation using 2,910 applications from the 2013 to 2017 cycles and a prospective validation using 2,715 applications during the 2018 application cycle were performed. To test the validated algorithm, a randomized trial was performed in the 2019 cycle, with 1,827 eligible applications being reviewed by faculty and 1,873 by algorithm. ResultsThe retrospective validation yielded area under the receiver operating characteristic (AUROC) values of 0.83, 0.64, and 0.83 and area under the precision-recall curve (AUPRC) values of 0.61, 0.54, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The prospective validation yielded AUROC values of 0.83, 0.62, and 0.82 and AUPRC values of 0.66, 0.47, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The randomized trial found no significant differences in overall interview recommendation rates according to faculty or algorithm and among female or underrepresented in medicine applicants. In underrepresented in medicine applicants, there were no significant differences in the rates at which the admissions committee offered an interview (70 of 71 in the faculty reviewer arm and 61 of 65 in the algorithm arm; P = .14). No difference in the rate of the committee agreeing with the recommended interview was found among female applicants (224 of 229 in the faculty reviewer arm and 220 of 227 in the algorithm arm; P = .55). ConclusionsThe virtual faculty screener algorithm successfully replicated faculty screening of medical school applications and may aid in the consistent and reliable review of medical school applicants.

show abstract

AI and Holistic Review

Cited by 24 publications

References 23 publications

Missing the missing values: The ugly duckling of fairness in machine learning

Missing the missing values: The ugly duckling of fairness in machine learning

Evaluating a Learned Admission-Prediction Model as a Replacement for Standardized Tests in College Admissions

Artificial Intelligence Screening of Medical School Applications: Development and Validation of a Machine-Learning Algorithm

Contact Info

Product

Resources

About