Bias in machine learning software: why? how? what to do?

Abstract

Increasingly, software is making autonomous decisions in case of criminal sentencing, approving credit cards, hiring employees, and so on. Some of these decisions show bias and adversely affect certain social groups (e.g. those defined by sex, race, age, marital status). Many prior works on bias mitigation take the following form: change the data or learners in multiple ways, then see if any of that improves fairness. Perhaps a better approach is to postulate root causes of bias and then applying some resolution strategy.This paper checks if the root causes of bias are the prior decisions about (a) what data was selected and (b) the labels assigned to those examples. Our Fair-SMOTE algorithm removes biased labels; and rebalances internal distributions so that, based on sensitive attribute, examples are equal in positive and negative classes. On testing, this method was just as effective at reducing bias as prior approaches. Further, models generated via Fair-SMOTE achieve higher performance (measured in terms of recall and F1) than other state-of-the-art fairness improvement algorithms.To the best of our knowledge, measured in terms of number of analyzed learners and datasets, this study is one of the largest studies on bias mitigation yet presented in the literature. CCS CONCEPTS• Software and its engineering → Software creation and management; • Computing methodologies → Machine learning.

Fairness Testing: A Comprehensive Survey and Analysis of Trends

“…ML software is developed following the data-driven programming paradigm. Therefore, data determine the decision logic of ML software to a large extent [17], and data bias is considered a main root cause of ML software bias [48]. Data testing aims to detect different types of data bias, including checking whether the labels of training data are biased (label bias) [35], whether the distribution of training data implies an unexpected correlation between the sensitive attribute and the outcome label (selection bias) [49], whether the features of training data contain bias (feature bias) [50].…”

“…For example, engineers cannot judge whether a system is fair to women if they are unaware of the outcomes that the system provides to men. In practice, metamorphic relations and statistical measurements are adopted to tackle the oracle problem of fairness testing [48], [55].…”

“…For example, for demographic parity, researchers calculate the favorable rate among different demographic groups and detect fairness violations by comparing these rates. If the rate difference, called Statistical Parity Difference (SPD) in the software fairness literature [35], [38], [48], [50], [118], is beyond a threshold, the software under test is identified as containing fairness bugs.…”

“…To mitigate label bias, Chakraborty et al [35], [48] leveraged situation testing to identify biased data points and remove them from the training data. Specifically, they divided the dataset into the privileged and unprivileged groups based on the sensitive attribute.…”

Developing a novel fair-loan-predictor through a multi-sensitive debiasing pipeline: DualFair

“…Reject option classification [24] is a post-processing strategy that translates favorable outcomes from the privileged group to the unprivileged group and unfavorable outcomes from the unprivileged group to the privileged group based upon a certain level of confidence and uncertainty. Chakraborty et al proposed Fair-SMOTE [25], a pre-processing and inprocessing approach, which balances class and label distributions and performs situation testing (i.e testing individual fairness through alternate "worlds").…”