xFAIR: Better Fairness via Model-based Rebalancing of Protected Attributes

Abstract

Machine learning software can generate models that inappropriately discriminate against specific protected social groups (e.g., groups based on gender, ethnicity, etc). Motivated by those results, software engineering researchers have proposed many methods for mitigating those discriminatory effects.While those methods are effective in mitigating bias, few of them can provide explanations on what is the cause of bias. Here we propose xFAIR, a model-based extrapolation method, that is capable of both mitigating bias and explaining the cause. In our xFAIR approach, protected attributes are represented by models learned from the other independent variables (and these models offer extrapolations over the space between existing examples). We then use the extrapolation models to relabel protected attributes, which aims to offset the biased predictions of the classification model via rebalancing the distribution of protected attributes.The experiments of this paper show that, without compromising (original) model performance, xFAIR can achieve significantly better group and individual fairness (as measured in different metrics) than benchmark methods. Moreover, when compared to another instance-based rebalancing method, our model-based approach shows faster runtime and thus better scalability.For replication purposes, we make our algorithm available at https://github.com/anonymous12138/biasmitigation.

Fairness Testing: A Comprehensive Survey and Analysis of Trends

“…Peng et al [144] used logistic regression and decision tree algorithms as models to extrapolate the correlations among dependent variables that might cause bias in training data.…”

“…Usage scenario Description Link FairTest [140] General ML software Analyzing associations between software outcomes and sensitive attributes [196] Themis [55] General ML software Black-box random discriminatory instance generation [197] Aequitas [78] General ML software Black-box search-based discriminatory instance generation [198] ExpGA [77] General ML software Black-box search-based discriminatory instance generation [199] fairCheck [89] General ML software Verification-based discriminatory instance generation [200] MLCheck [88] General ML software Verification-based discriminatory instance generation [201] LTDD [50] General ML software Detecting which data features and which parts of them are biased [202] Fair-SMOTE [48] General ML software Detecting biased data labels and data distributions [203] xFAIR [144] General ML software Extrapolation of correlations among data features that might cause bias [204] Fairway [35] General ML software Detecting biased data labels and optimal hyper-parameters for ML fairness [205] Parfait-ML [46] General ML software Searching for hyper-parameters optimal to ML software fairness [206] Fairea [38] General ML software Testing fairness repair algorithms [207] IBM AIF360 [161] General ML software Examining and mitigating discrimination and bias in ML software [119] scikit-fairness [208] General ML software Examining and mitigating discrimination and bias in ML software [208] LiFT [209] General ML software Examining and mitigating discrimination and bias in ML software [210] SageMaker Clarify [211] General ML software Measuring bias that occurs in each stage of the ML life cycle [212] FairVis [213] General ML software Visual analytics for discovering intersectional bias in ML software [214] FairRepair [155] Tree-based classifiers Detecting paths responsible for unfairness in tree-based classifiers [215] ADF…”

Studying the Practices of Deploying Machine Learning Projects on Docker

“…The sole idea of the above approach is to analyze and mitigate the bias defined by the protected subgroup of these sen-sitive features to estimate the direct discrimination. Similarly other previous studies have mainly relied upon simple statistical analysis involving association or correlation measures [16,48,70,51,60]. However, such analyses can lead to incorrect conclusions because they largely ignore the effect of confounding variables;-variables that can be used to determine both the outcome and the feature pairs.…”

Law Modeling for Fairness Requirements Elicitation in Artificial Intelligence Systems

“…On the other hand, several techniques are focused in modifying the dataset according to protected data in order to mitigate bias, mainly based on rebalancing techniques [11] [15]. [9] try to identify bias in the labels and proposes a method based on the re-weighting of the elements in the dataset to mitigate such bias.…”