"Ignorance and Prejudice" in Software Fairness

Abstract

Machine learning software can be unfair when making human-related decisions, having prejudices over certain groups of people. Existing work primarily focuses on proposing fairness metrics and presenting fairness improvement approaches. It remains unclear how key aspect of any machine learning system, such as feature set and training data, affect fairness. This paper presents results from a comprehensive study that addresses this problem. We find that enlarging the feature set plays a significant role in fairness (with an average effect rate of 38%). Importantly, and contrary to widely-held beliefs that greater fairness often corresponds to lower accuracy, our findings reveal that an enlarged feature set has both higher accuracy and fairness. Perhaps also surprisingly, we find that a larger training data does not help to improve fairness. Our results suggest a larger training data set has more unfairness than a smaller one when feature sets are insufficient; an important cautionary finding for practising software engineers.

A social and technical sustainability requirements catalogue

“…For example, a decisionmaking system should automatically measure discrimination (e.g., to ensure fairness in software crowdsourcing platforms). Fairness is considered important in software by other authors [37][38][39][40].…”

Fairness Testing: A Comprehensive Survey and Analysis of Trends

“…For example, for demographic parity, researchers calculate the favorable rate among different demographic groups and detect fairness violations by comparing these rates. If the rate difference, called Statistical Parity Difference (SPD) in the software fairness literature [35], [38], [48], [50], [118], is beyond a threshold, the software under test is identified as containing fairness bugs.…”

“…Feature bias occurs when some features in the training data are highly related to the sensitive attribute, and these correlated features can thus become the root cause of software unfairness [50]. Zhang et al [118] explored how the feature set influences ML software fairness. The results showed that the feature set plays a significant role in fairness, which motivates the fairness work about testing data features.…”

“…Figure 9 examines the distribution of different fairness categories in the fairness testing literature. Previous work [27], [38], [118] observed that group fairness and individual fairness were the most widely studied in the literature. In line with these observations, we find that these two categories account for a total of 93% of fairness testing work.…”

Did You Do Your Homework? Raising Awareness on Software Fairness and Discrimination

“…In 2018, Brun and Meliou [9] stated that ensuring the fairness of software systems (software fairness) is a critical software engineering problem to be tackled from multiple directions, and since then it has gained more and more attention from software engineering research [10], [11], [12], [13], [14], [15].…”