A Statistical Test for Probabilistic Fairness

Taşkesen, Bahar; Blanchet, José; Kühn, Daniel; Nguyên, Viêt Anh

doi:10.1145/3442188.3445927

Cited by 23 publications

(23 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the continuous estimating function, it is optimal to move every point O p (N −1/2 ) distances, which results in a O p (N −1/2 ) convergence rate. Therefore, let us emphasize again the key qualitative difference between our contributions and those of Taskesen et al (2021). A statistical noise gives the empirical appearance of unfairness in two ways: (A) small statistical fluctuations around all data points; (B) a small sub-population with large outcome fluctuations around the decision boundary.…”

Section: The Structure Of the Wasserstein Projectionmentioning

confidence: 83%

“…This statistical phenomenon is due to the discontinuity of the estimating function C(X)φ(U, E Q [U ]) in X, where the transporter is able to move a small amount of probability mass, but the move results in a significant change of the value for the estimating function around the discontinuity region. The O p (N −1 ) convergence rate is in contrast to the rate in Blanchet et al (2019), Taskesen et al (2021) and Cisneros-Velarde et al (2020), where the estimating function is assumed to be continuous. For the continuous estimating function, it is optimal to move every point O p (N −1/2 ) distances, which results in a O p (N −1/2 ) convergence rate.…”

Section: The Structure Of the Wasserstein Projectionmentioning

confidence: 95%

“…Recently, optimal transport divergences (Villani, 2008) become an attractive tool in many recent machine learning studies, including missing data imputation, geodesic PCA, point embeddings, and repairing data with Wasserstein barycenters for training fair classifiers (Silvia et al, 2020;Gordaliza et al, 2019;Zehlike et al, 2020). Taskesen et al (2021) uses optimal transport projections to test a smooth relaxation of the equal opportunity criterion called probabilistic fairness; see Pleiss et al (2017). This relaxation is required in Taskesen et al (2021) to overcome the discontinuities in the classification boundaries which create technical complications when computing the optimal transport projections.…”

Section: Introductionmentioning

confidence: 99%

“…Taskesen et al (2021) uses optimal transport projections to test a smooth relaxation of the equal opportunity criterion called probabilistic fairness; see Pleiss et al (2017). This relaxation is required in Taskesen et al (2021) to overcome the discontinuities in the classification boundaries which create technical complications when computing the optimal transport projections. Further, the resulting test statistic involves a non-convex optimization problem which is difficult to compute.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Testing Group Fairness via Optimal Transport Projections

Si,

Murthy,

Blanchet

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We present a statistical testing framework to detect if a given machine learning classifier fails to satisfy a wide range of group fairness notions. The proposed test is a flexible, interpretable, and statistically rigorous tool for auditing whether exhibited biases are intrinsic to the algorithm or due to the randomness in the data. The statistical challenges, which may arise from multiple impact criteria that define group fairness and which are discontinuous on model parameters, are conveniently tackled by projecting the empirical measure onto the set of group-fair probability models using optimal transport. This statistic is efficiently computed using linear programming and its asymptotic distribution is explicitly obtained. The proposed framework can also be used to test for testing composite fairness hypotheses and fairness with multiple sensitive attributes. The optimal transport testing formulation improves interpretability by characterizing the minimal covariate perturbations that eliminate the bias observed in the audit.

show abstract

Section: The Structure Of the Wasserstein Projectionmentioning

confidence: 83%

Section: The Structure Of the Wasserstein Projectionmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Testing Group Fairness via Optimal Transport Projections

Si,

Murthy,

Blanchet

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Given a logistic classifier parametrized by θ, the decision to reject the probabilistic fairness of this classifier now relies on computing the projection distance P(P n , θ) and on computing the empirical quantile estimate of βχ 2 1 [88]. The limit result in Theorem 2 relies on the assumption that h is continuously differentiable.…”

Section: Statistical Hypothesis Testing With Projection Based Profile...mentioning

confidence: 99%

Statistical Analysis of Wasserstein Distributionally Robust Estimators

Blanchet¹,

Murthy²,

Nguyen³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We consider statistical methods which invoke a min-max distributionally robust formulation to extract good out-of-sample performance in data-driven optimization and learning problems. Acknowledging the distributional uncertainty in learning from limited samples, the min-max formulations introduce an adversarial inner player to explore unseen covariate data. The resulting Distributionally Robust Optimization (DRO) formulations, which include Wasserstein DRO formulations (our main focus), are specified using optimal transportation phenomena. Upon describing how these infinite-dimensional min-max problems can be approached via a finite-dimensional dual reformulation, the tutorial moves into its main component, namely, explaining a generic recipe for optimally selecting the size of the adversary's budget. This is achieved by studying the limit behavior of an optimal transport projection formulation arising from an inquiry on the smallest confidence region that includes the unknown population risk minimizer. Incidentally, this systematic prescription coincides with those in specific examples in high-dimensional statistics and results in error bounds that are free from the curse of dimensions. Equipped with this prescription, we present a central limit theorem for the DRO estimator and provide a recipe for constructing compatible confidence regions that are useful for uncertainty quantification. The rest of the tutorial is devoted to insights into the nature of the optimizers selected by the min-max formulations and additional applications of optimal transport projections.

show abstract

Semi-discrete optimal transport: hardness, regularization and numerical solution

Taşkesen

Shafieezadeh-Abadeh²,

Kühn

2022

Math. Program.

Self Cite

View full text Add to dashboard Cite

Semi-discrete optimal transport problems, which evaluate the Wasserstein distance between a discrete and a generic (possibly non-discrete) probability measure, are believed to be computationally hard. Even though such problems are ubiquitous in statistics, machine learning and computer vision, however, this perception has not yet received a theoretical justification. To fill this gap, we prove that computing the Wasserstein distance between a discrete probability measure supported on two points and the Lebesgue measure on the standard hypercube is already $$\#$$ # P-hard. This insight prompts us to seek approximate solutions for semi-discrete optimal transport problems. We thus perturb the underlying transportation cost with an additive disturbance governed by an ambiguous probability distribution, and we introduce a distributionally robust dual optimal transport problem whose objective function is smoothed with the most adverse disturbance distributions from within a given ambiguity set. We further show that smoothing the dual objective function is equivalent to regularizing the primal objective function, and we identify several ambiguity sets that give rise to several known and new regularization schemes. As a byproduct, we discover an intimate relation between semi-discrete optimal transport problems and discrete choice models traditionally studied in psychology and economics. To solve the regularized optimal transport problems efficiently, we use a stochastic gradient descent algorithm with imprecise stochastic gradient oracles. A new convergence analysis reveals that this algorithm improves the best known convergence guarantee for semi-discrete optimal transport problems with entropic regularizers.

show abstract

A Statistical Test for Probabilistic Fairness

Cited by 23 publications

References 36 publications

Testing Group Fairness via Optimal Transport Projections

Testing Group Fairness via Optimal Transport Projections

Statistical Analysis of Wasserstein Distributionally Robust Estimators

Semi-discrete optimal transport: hardness, regularization and numerical solution

Contact Info

Product

Resources

About