FLEA: Provably Fair Multisource Learning from Unreliable Training Data

Abstract

Fairness-aware learning aims at constructing classifiers that not only make accurate predictions, but do not to discriminate against specific groups. It is a fast-growing area of machine learning with far-reaching societal impact. However, existing fair learning methods are vulnerable to accidental or malicious artifacts in the training data, which can cause them to unknowingly produce unfair classifiers. In this work we address the problem of fair learning from unreliable training data in the robust multisource setting, where the available training data comes from multiple sources, a fraction of which might be not representative of the true data distribution. We introduce FLEA, a filtering-based algorithm that allows the learning system to identify and suppress those data sources that would have a negative impact on fairness or accuracy if they were used for training. We show the effectiveness of our approach by a diverse range of experiments on multiple datasets. Additionally we prove formally that -given enough data-FLEA protects the learner against unreliable data as long as the fraction of affected data sources is less than half.* Equal contribution Preprint. Under review.

Improving fairness generalization through a sample-robust optimization method

“…Furthermore, fairness robustness has also been studied in other settings, such as multi-source learning [26], or for other notions of fairness such as individual fairness [48]. Both are out of the scope of this paper and thus we do not further detail these approaches.…”

“…5. Results for the remaining metrics are detailed here.Figures 21, 22, 23, 24 summarize the experimental results using the Statistical Parity metric.Figures 25,26,27,28 summarize the experimental results using the Predictive Equality metric.Figures 29,30,31,32 summarize the experimental results using the Equalized Odds metric.C.5 Comparison between the Exact and Heuristic Methods: Fairness Sample RobustnessThis appendix section contains the experimental comparison of the fairness sample robustness of our exact and heuristic methods with FairCORELS. Results for the Statistical Parity metric are presented in Section 5.1.6.…”