CertiFair: A Framework for Certified Global Fairness of Neural Networks

Abstract

We consider the problem of whether a Neural Network (NN) model satisfies global individual fairness. Individual Fairness (defined in [1]) suggests that similar individuals with respect to a certain task are to be treated similarly by the decision model. In this work, we have two main objectives. The first is to construct a verifier which checks whether the fairness property holds for a given NN in a classification task or provide a counterexample if it is violated, i.e., the model is fair if all similar individuals are classified the same, and unfair if a pair of similar individuals are classified differently. To that end, We construct a sound and complete verifier that verifies global individual fairness properties of ReLU NN classifiers using distance-based similarity metrics. The second objective of this paper is to provide a method for training provably fair NN classifiers from unfair (biased) data. We propose a fairness loss that can be used during training to enforce fair outcomes for similar individuals. We then provide provable bounds on the fairness of the resulting NN. We run experiments on commonly used fairness datasets that are publicly available and we show that global individual fairness can be improved by 96 % without significant drop in test accuracy.Preprint. Under review.

Beyond robustness: Resilience verification of tree-based classifiers

“…This is important in the fairness setting, because fairness is particularly relevant for minorities for which it might be hard to collect representative data in the test set. Indeed, the need for global fairness verification has been recently advocated for neural networks [18].…”

“…Indeed, the first work on the verification of individual fairness focused on linear models such as logistic regression and support vector machines [11]. Most later work, instead, focused on neural networks and cannot be applied to decision tree ensembles [18], [12], [10]. A general approach to fairness verification based on SMT solving was presented in [13].…”

“…This is sub-optimal because it does not allow one to prove that unfair behavior can never affect specific classes of individuals. For this reason, recent papers advocated the adoption of formal fairness verification techniques to prove lack of discrimination [10], [11], [12], [13], [18]. Most of the work in the area, however, just focuses on deep neural networks and disregards tree-based ML models, such as decision trees [19] and random forests [20], which are still exceptionally popular, in particular for nonperceptual classification tasks.…”

“…The only notable fairness verification approach designed for tree-based classifiers leverages abstract interpretation to verify local fairness properties [9]. Unfortunately, local fairness is now recognized as a rather weak property predicating just on specific test instances, while global fairness predicates over all the possible inputs of the classifier and is thus more reliable to assess the actual fairness guarantees that it provides [18].…”

REGLO: Provable Neural Network Repair for Global Robustness Properties

“…The first two authors contributed equally to this paper. cess (Leino, Wang, and Fredrikson 2021;Chen et al 2021;Wadsworth, Vera, and Piech 2018;Celis and Keswani 2019;Adel et al 2019;Tao et al 2022;Khedr and Shoukry 2022). These approaches mainly involve introducing a loss function related to global robustness such that global robustness is taken into account during gradient descent.…”