Not So Fair: The Impact of Presumably Fair Machine Learning Models

Jorgensen, Mackenzie; Richert, Hannah; Black, Elizabeth; Criado, Natalia; Such, Jose

doi:10.1145/3600211.3604699

Cited by 4 publications

(2 citation statements)

References 28 publications

(51 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This model shows that constraining selection policies to satisfy certain fairness criteria, such as demographic parity or equal opportunity, can sometimes lead to greater declines in qualification status compared to the unconstrained policy [64]. Jorgensen et al extend this result to several other fairness criteria [51]. Williams and Kolter model a similar loan approval setting as Liu et al, with slight modifications to the update function [91].…”

Section: Related Workmentioning

confidence: 92%

Structural Interventions and the Dynamics of Inequality

Zhang,

Hosoi

2024

The 2024 ACM Conference on Fairness, Accountability, and Transparency

View full text Add to dashboard Cite

Recent conversations in the algorithmic fairness literature have raised several concerns with standard conceptions of fairness. First, constraining predictive algorithms to satisfy fairness benchmarks may sometimes lead to non-optimal outcomes for disadvantaged groups. Second, technical interventions are often ineffective by themselves, especially when divorced from an understanding of structural processes that generate social inequality. Inspired by both these critiques, we construct a common decision-making model, using mortgage loans as a running example. We show that under some conditions, any choice of decision threshold will inevitably perpetuate existing disparities in financial stability unless one deviates from the Pareto optimal policy. This confirms the intuition that technical interventions, such as fairness constraints, often do not sufficiently address persistent underlying inequities. Then, we model the effects of three different types of interventions: (1) policy changes in the algorithm's decision threshold, and external changes to parameters that govern the downstream effects of late payment for (2) the whole population or (3) disadvantaged subgroups. We show how different interventions are recommended depending on the difficulty of enacting structural change upon external parameters and depending on the policymaker's preferences for equity or efficiency. Counterintuitively, we demonstrate that preferences for efficiency over equity may sometimes lead to recommendations for interventions that target the under-resourced group alone. Finally, we simulate the effects of interventions on a dataset that combines HMDA and Fannie Mae loan data. This research highlights the ways that structural inequality can be perpetuated by seemingly unbiased decision mechanisms, and it shows that in many situations, technical solutions must be paired with external, context-aware interventions to enact social change. CCS CONCEPTS• Applied computing → Sociology; • Human-centered computing → Collaborative and social computing design and evaluation methods; Social engineering (social sciences).

show abstract

Section: Related Workmentioning

confidence: 92%

Structural Interventions and the Dynamics of Inequality

Zhang,

Hosoi

2024

The 2024 ACM Conference on Fairness, Accountability, and Transparency

View full text Add to dashboard Cite

show abstract

“…However, theoretical results state that it is impossible to satisfy different fairness notions at the same time (Chouldechova, 2017;Kleinberg et al, 2017). Not only fairness notions are in tension among each other (Alves et al, 2023), but also with other quality requirements of AI systems, such as predictive accuracy (Menon & Williamson, 2018), calibration (Pleiss et al, 2017), impact (Jorgensen et al, 2023), and privacy (Cummings et al, 2019), for which Pareto optimality should be considered (Wei & Niethammer, 2022). Moreover, the choice of a fairness metric requires to take into account several contrasting objectives: stakeholders' utility, human value alignment (Friedler et al, 2021), people's actual perception of fairness (Saha et al, 2020;Srivastava et al, 2019), and legal and normative constraints (Xenidis, 2020;Kroll et al, 2017).…”

Section: Using Fair-ai With a Guidancementioning

confidence: 99%

Policy advice and best practices on bias and fairness in AI

Alvarez,

Colmenarejo,

Elobaid

et al. 2024

Ethics Inf Technol

View full text Add to dashboard Cite

The literature addressing bias and fairness in AI models (fair-AI) is growing at a fast pace, making it difficult for novel researchers and practitioners to have a bird’s-eye view picture of the field. In particular, many policy initiatives, standards, and best practices in fair-AI have been proposed for setting principles, procedures, and knowledge bases to guide and operationalize the management of bias and fairness. The first objective of this paper is to concisely survey the state-of-the-art of fair-AI methods and resources, and the main policies on bias in AI, with the aim of providing such a bird’s-eye guidance for both researchers and practitioners. The second objective of the paper is to contribute to the policy advice and best practices state-of-the-art by leveraging from the results of the NoBIAS research project. We present and discuss a few relevant topics organized around the NoBIAS architecture, which is made up of a Legal Layer, focusing on the European Union context, and a Bias Management Layer, focusing on understanding, mitigating, and accounting for bias.

show abstract