Fair Allocation through Selective Information Acquisition

Cai, William; Gaebler, Johann; Garg, Nikhil; Goel, Sharad

doi:10.1145/3375627.3375823

Cited by 21 publications

(20 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fairness in machine learning and mechanism design. Recent machine learning work applies fairness notions to college admissions and related allocation problems (Cai et al, 2020;Emelianov et al, 2020;Faenza et al, 2020;Hu et al, 2019;Kannan et al, 2019;Liu et al, 2020;Mouzannar et al, 2019).…”

Section: Economics Of Discriminationmentioning

confidence: 99%

Standardized Tests and Affirmative Action

Garg

Monachou

2021

Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

Self Cite

View full text Add to dashboard Cite

The University of California recently suspended through 2024 the requirement that California applicants submit SAT scores, upending the major role standardized testing has played in college admissions. We study the impact of this decision and its interplay with other policies-such as affirmative action-on admitted class composition.We develop a market model with schools and students. Students have an unobserved true skill level, a potentially observed demographic group membership, and an observed application with both test scores and other features. Bayesian schools optimize the dual-objectives of admitting (1) the "most qualified" and (2) a "diverse" cohort. They estimate each applicant's true skill level using the observed features and potentially their group membership, and then admit students with or without affirmative action.We show that dropping test scores may exacerbate disparities by decreasing the amount of information available for each applicant. However, if there are substantial barriers to testing, removing the test improves both academic merit and diversity by increasing the size of the applicant pool. We also find that affirmative action alongside using group membership in skill estimation is an effective strategy with respect to the dual-objective. Findings are validated with calibrated simulations using cross-national testing data.

show abstract

Section: Economics Of Discriminationmentioning

confidence: 99%

Standardized Tests and Affirmative Action

Garg

Monachou

2021

Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

Self Cite

View full text Add to dashboard Cite

show abstract

“…Abdulkadiroğlu and Sönmez [2003] and Foster and Vohra [1992]. Our work is more closely connected to recent work in both the machine learning and mechanism design communities on fairness in ranking and allocation in general, and on education in particular [Cai, Gaebler, Garg, and Goel, 2020, Emelianov, Gast, Gummadi, and Loiseau, 2020, Faenza, Gupta, and Zhang, 2020, Kannan, Roth, and Ziani, 2019, Haghtalab, Immorlica, Lucier, and Wang, 2020, Immorlica, Ligett, and Ziani, 2019, Kleinberg and Mullainathan, 2019, Liu, Wilson, Haghtalab, Kalai, Borgs, and Chayes, 2020, Mouzannar, Ohannessian, and Srebro, 2019. For a review of fairness in both machine learning and mechanism design, see Finocchiaro, Maio, Monachou, Patro, Raghavan, Stoica, and Tsirtsis [2021].…”

Section: Related Workmentioning

confidence: 79%

Test-optional Policies: Overcoming Strategic Behavior and Informational Gaps

Zhi

Garg

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Due to the Covid-19 pandemic, more than 500 US-based colleges and universities went "testoptional" for admissions and promised that they would not penalize applicants for not submitting test scores, part of a longer trend to rethink the role of testing in college admissions. However, it remains unclear how (and whether) a college can simultaneously use test scores for those who submit them, while not penalizing those who do not-and what that promise even means. We formalize these questions, and study how a college can overcome two challenges with optional testing: strategic applicants (when those with low test scores can pretend to not have taken the test), and informational gaps (it has more information on those who submit a test score than those who do not). We find that colleges can indeed do so, if and only if they are able to use information on who has test access and are willing to randomize admissions.

show abstract

“…We see our work as part of a broadening of how machine learning practitioners operationalize algorithmic fairness. In addition to approaches tailored to improving the equity of models trained on static datasets, it is important to consider issues that arise at various stages of the training and deployment of statistical models, including constructing equitable training sets [27,27,31,31,44,44,51,51], interventions to bolster model performance for traditionally underserved groups, such as screening [6,13,49], and designing more equitable interventions given a set of risk scores [17]. We hope our work will help support these ongoing efforts.…”

Section: Discussionmentioning

confidence: 99%

Adaptive Sampling Strategies to Construct Equitable Training Datasets

Cai¹,

Encarnacion²,

Chern³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In domains ranging from computer vision to natural language processing, machine learning models have been shown to exhibit stark disparities, often performing worse for members of traditionally underserved groups. One factor contributing to these performance gaps is a lack of representation in the data the models are trained on. It is often unclear, however, how to operationalize representativeness in specific applications. Here we formalize the problem of creating equitable training datasets, and propose a statistical framework for addressing this problem. We consider a setting where a model builder must decide how to allocate a fixed data collection budget to gather training data from different subgroups. We then frame dataset creation as a constrained optimization problem, in which one maximizes a function of group-specific performance metrics based on (estimated) group-specific learning rates and costs per sample. This flexible approach incorporates preferences of model-builders and other stakeholders, as well as the statistical properties of the learning task. When data collection decisions are made sequentially, we show that under certain conditions this optimization problem can be efficiently solved even without prior knowledge of the learning rates. To illustrate our approach, we conduct a simulation study of polygenic risk scores on synthetic genomic data-an application domain that often suffers from non-representative data collection.We find that our adaptive sampling strategy outperforms several common data collection heuristics, including equal and proportional sampling, demonstrating the value of strategic dataset design for building equitable models. CCS Concepts: • Computing methodologies → Machine learning; Artificial intelligence; • Theory of computation → Design and analysis of algorithms.

show abstract

Fair Allocation through Selective Information Acquisition

Cited by 21 publications

References 17 publications

Standardized Tests and Affirmative Action

Standardized Tests and Affirmative Action

Test-optional Policies: Overcoming Strategic Behavior and Informational Gaps

Adaptive Sampling Strategies to Construct Equitable Training Datasets

Contact Info

Product

Resources

About