Stable and Fair Classification

Abstract

Fair classification has been a topic of intense study in machine learning, and several algorithms have been proposed towards this important task. However, in a recent study, Friedler et al. observed that fair classification algorithms may not be stable with respect to variations in the training dataset -a crucial consideration in several real-world applications. Motivated by their work, we study the problem of designing classification algorithms that are both fair and stable. We propose an extended framework based on fair classification algorithms that are formulated as optimization problems, by introducing a stability-focused regularization term. Theoretically, we prove a stability guarantee, that was lacking in fair classification algorithms, and also provide an accuracy guarantee for our extended framework. Our accuracy guarantee can be used to inform the selection of the regularization parameter in our framework. To the best of our knowledge, this is the first work that combines stability and fairness in automated decision-making tasks. We assess the benefits of our approach empirically by extending several fair classification algorithms that are shown to achieve the best balance between fairness and accuracy over the Adult dataset. Our empirical results show that our framework indeed improves the stability at only a slight sacrifice in accuracy.

When Fair Ranking Meets Uncertain Inference

“…Researchers in academia and industry are building tools to help detect biases in algorithms [6,52,65,77]. Additionally, there is a growing library of fairness-aware machine learning algorithms for classification [31,38,43,57], regression [2,7], causal inference [51,60], word embeddings [11,12], machine translation [25] and finally, ranking [18,73,81].…”

Survey: Leakage and Privacy at Inference Time

“…Most proposed methods treat solving for fairness based on the definition of fairness tailored to their specific objective. Of considerable importance are techniques such as those proposed by [220] which not only satisfy fairness constraints, but also tend to be stable towards adversarial attacks and variations in datasets during testing. Regression-based fairness techniques eliminate bias at training time by hand-crafting loss functions that conform to group fairness, individual fairness or hybrid fairness, although they have not received a lot of attention in research [221].…”

FairCanary: Rapid Continuous Explainable Fairness

“…The use of algorithms to aid critical decision making processes in the government and the industry has attracted commensurate scrutiny from academia, lawmakers and social justice workers in recent times [4,7,71], because ML systems trained on a snapshot of the society has the unintended consequences of learning, propagating and amplifying historical social biases and power dynamics [5,56]. The current research landscape consists of both ML explanation methods and fairness metrics to try and uncover the problems of trained models [8,30,45,59,68], and fairness aware ML algorithms, for instance classification [31,34,37,47], regression [2,9], causal inference [43,49], word embeddings [13,14] and ranking [16,64,72].…”