Multiaccurate Proxies for Downstream Fairness

Abstract

We study the problem of training a model that must obey demographic fairness conditions when the sensitive features are not available at training time -in other words, how can we train a model to be fair by race when we don't have data about race? We adopt a fairness pipeline perspective, in which an "upstream" learner that does have access to the sensitive features will learn a proxy model for these features from the other attributes. The goal of the proxy is to allow a general "downstream" learnerwith minimal assumptions on their prediction task -to be able to use the proxy to train a model that is fair with respect to the true sensitive features. We show that obeying multiaccuracy constraints with respect to the downstream model class suffices for this purpose, and provide sample-and oracle efficientalgorithms and generalization bounds for learning such proxies. In general, multiaccuracy can be much easier to satisfy than classification accuracy, and can be satisfied even when the sensitive features are hard to predict.

Distributionally Robust Data Join

“…However, Awasthi et al [2021] show that due to different underlying base rates across groups, the Bayes optimal predictor for the demographic group information can result in maximally biased estimate of unfairness. Diana et al [2021] demonstrate that one can rely on a multi-accurate regressor, which was first introduced by Kim et al [2019], as opposed to a 0-1 classifier in order to estimate the unfairness without any bias and also build a fair classifier for downstream tasks. When only some data points are missing demographic information, Jeong et al [2021] show how to bypass the need to explicitly impute the missing values and instead rely on some decision tree based approach in order to optimize a fairness-regularized objective function.…”

Simple and near-optimal algorithms for hidden stratification and multi-group learning

“…Given a class of groups G, multiaccuracy requires that the expectation of a predictor is close to expectation of y conditioned on any g ∈ G (Hébert-Johnson et al, 2018;Diana et al, 2021). Kim et al (2019) showed that for an appropriate choice of groups, multiaccuracy implies a type of multi-group learnability result.…”