What's Fair about Individual Fairness?

Abstract

One of the main lines of research in algorithmic fairness involves individual fairness (IF) methods. Individual fairness is motivated by an intuitive principle I call "similar treatment," which requires that similar individuals be treated similarly. IF offers a precise account of this definition using distance metrics to evaluate the similarity of individuals. Proponents of individual fairness have argued that it gives the correct definition of algorithmic fairness, and that it should therefore be preferred to other methods for determining fairness. I argue that individual fairness cannot serve as a definition of fairness. Moreover, IF methods should not be given priority over other fairness methods, nor used in isolation from them. To support these conclusions, I describe four in-principle problems for individual fairness as a definition and as a method for ensuring fairness: (1) counterexamples show that similar treatment (and therefore IF) are insufficient to guarantee fairness; (2) IF methods for learning similarity metrics are at risk of encoding human implicit bias; (3) IF requires prior moral judgments, limiting its usefulness as a guide for fairness and undermining its claim to define fairness; and (4) the incommensurability of relevant moral values makes similarity metrics impossible for many tasks. In light of these limitations, I suggest that individual fairness cannot be a definition of fairness, and instead should be seen as one tool among many for ameliorating algorithmic bias.

Gerrymandering Individual Fairness

“…This is a contraction because it reduces the distance between all predictions to 0. The fact that constant predictors are Lipschitz was already noted in Dwork et al (2012) and also in Fleisher (2021) in the form of Universal Rejection: If a score measures suitability for college, it is IF to assign the same, low score to all applicants, and thus reject them all. Fleisher notes that this may be considered unfair to those who would be suitable for college, but are rejected.…”

“…But this does not resolve the issue: It is equally unclear why it is a good idea to "express our awareness" of fairness through the metric structure of a prediction problem. Note that IF has other problems that have been pointed out in the literature, such as the problem of incommensurability pointed out by Fleisher (2021), viz. that IF requires that there need to be distances between all individuals.…”

“…3.1. ); Fleisher (2021). The choice of X as maximally nondiscriminative corresponds to a minimally individualized notion of fairness: If we all have the same features, "We're all alike" (Friedler et al, 2016) and thus should be treated similarly (or equally).…”

“…For example, if we want "individualized justice", i.e., predictions that depend on the features of individuals , then X needs to be sufficiently fine-grained to represent unique identifiers for individuals. The same goes for Y and Ŷ , which may need to be appropriately structured to make room for abstention, or incommensurable predictions, such that, e.g., Ŷ is a partial orders; see Fleisher (2021) for more on why this may be desirable.…”

Survey on Fair Reinforcement Learning: Theory and Practice

“…On the other hand, Fleisher [41] present the following critiques of individual fairness -i) counterexamples show that similar treatment guaranteed by individual fairness is insufficient to guarantee fairness, ii) the used similarity metrics/arbiters may suffer from implicit systematic biases, iii) it cannot offer a substantive, non-circular definition of fairness because determining which features are task-relevant and thus apt for measuring similarity requires making moral judgments about what fairness constitutes, and iv) if incommensurable moral values are relevant for determining similarity for a task, similarity cannot be represented as a distance metric.…”