Counterfactual Fairness: Unidentification, Bound and Algorithm

Abstract

Fairness-aware learning studies the problem of building machine learning models that are subject to fairness requirements. Counterfactual fairness is a notion of fairness derived from Pearl's causal model, which considers a model is fair if for a particular individual or group its prediction in the real world is the same as that in the counterfactual world where the individual(s) had belonged to a different demographic group. However, an inherent limitation of counterfactual fairness is that it cannot be uniquely quantified from the observational data in certain situations, due to the unidentifiability of the counterfactual quantity. In this paper, we address this limitation by mathematically bounding the unidentifiable counterfactual quantity, and develop a theoretically sound algorithm for constructing counterfactually fair classifiers. We evaluate our method in the experiments using both synthetic and real-world datasets, as well as compare with existing methods. The results validate our theory and show the effectiveness of our method.

PreCoF: Counterfactual Explanations for Fairness

“…In recent years, fairness-aware machine learning has been studied from the causal perspective using causal modelling (Pearl et al (2000)). In line with this research, Kusner et al (2017) defines Counterfactual Fairness as a notion of fairness derived from Pearl's causal model (Pearl et al (2000)) where for an individual the prediction of the model is considered as fair if it is the same in the real world as it would be if the individual would belong to a different demographic group (Kusner et al (2017); Wu, Zhang, and Wu (2019)).…”

Survey on Causal-based Machine Learning Fairness Notions

“…The identifiability of causal quantities has been extensively studied in the literature: causal effect (intervention) identification [11,14,24,32,33,[35][36][37], counterfactual identification [30][31][32]38], direct/indirect effects [23] and path-specific effect identification [1,20,30,42,43]. This section summarizes the main identifiability conditions as they relate to the specific problem of discrimination discovery.…”

“…In Markovian models, the identifiability of counterfactual fairness and individual equality of effort depends on the identifiability of the term ( 1 |X = x, = 0 ) which is only identifiable if X does not contain any variable which is at the same time descendant of and ancestor of , that is, X ∩ B = ∅ where B = ( ) ∩ ( ) [38]. Path-specific counterfactual fairness is applicable provided that the model is Markovian and the recanting witness criterion is not satisfied (Section 5.4).…”

Fair and Effective Policing for Neighborhood Safety: Understanding and Overcoming Selection Biases

“…The bias can be formulated as the risk discrepancy between the empirical risk and the true risk, which can be solved in a re-weighting formulation. To better understand the meaning of discrepancy, inspired by counterfactual modeling Pearl et al (2009) ; Pearl (2010) and causal inference Kallus (2020) ; Hernán and Robins (2010) ; Khademi et al (2019) , we hypothesize the counterfactual distribution Wu et al (2019a) of each driver, and then show that a supervised estimator can be unbiased only when selected probability, i.e., the searched probability of drivers by the police, is known and fixed. To sum up, selection bias accounts to risk discrepancy.…”

“…Inspired by causal inference Pearl (2010) , we resort to counterfactual modeling to solve the second challenge. Regarding searched/unsearched action as a treatment/control intervention Wu et al (2019a) respectively, the core idea in causal inference is to create a pseudo population where the distributions of the treated group and control group are similar. So the outcome is independent with treatment conditional on the confounder.…”