The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric

Abstract

Where machine-learned predictive risk scores inform high-stakes decisions, such as bail and sentencing in criminal justice, fairness has been a serious concern. Recent work has characterized the disparate impact that such risk scores can have when used for a binary classification task. This may not account, however, for the more diverse downstream uses of risk scores and their non-binary nature. To better account for this, in this paper, we investigate the fairness of predictive risk scores from the point of view of a bipartite ranking task, where one seeks to rank positive examples higher than negative ones. We introduce the xAUC disparity as a metric to assess the disparate impact of risk scores and define it as the difference in the probabilities of ranking a random positive example from one protected group above a negative one from another group and vice versa. We provide a decomposition of bipartite ranking loss into components that involve the discrepancy and components that involve pure predictive ability within each group. We use xAUC analysis to audit predictive risk scores for recidivism prediction, income prediction, and cardiac arrest prediction, where it describes disparities that are not evident from simply comparing within-group predictive performance.

Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems

“…This pairwise ranking accuracy has a nice connection with the Mann-Whitney U-test [29], and aligns well with the equality gap metric [8] and the xAUC metric [24] for classification.…”

“…The composed system fairness metric is evaluated on the order (rank) of the product of the component scores. Evaluating the fairness metric on rank order aligns well with most real-world multi-model recommender systems, but can also be applied to classification [24,8,33].…”

“…It focuses only on "advantaged" outcome Y = 1. More recently, metrics have been explored continuous scores from a classifier: [24,8,33] all break down AUC of these scores into Mann-Whitney U-tests [29].…”

Pairwise Fairness for Ranking and Regression

“…There have also been two recent papers on pairwise fairness notions [4,19], both of which are focused on ranking with categorical groups and are methodologically different from us.…”

“…Lastly, there are major experimental differences: they provide an in-depth case study of one real-world recommendation problem, whereas we provide a broad set of experiments on public and real-world data illustrating the effectiveness on both ranking and regression problems, for categorical or continuous protected attributes. [19] also provide pairwise fairness metrics based on AUC for bipartite ranking problems, but only consider categorical groups and propose a post-processing approach that fits a monotonic transform to an existing ranking model to optimize the proposed metrics. In contrast, we additionally handle regression problems and continuous protected attributes, and develop more flexible approaches that directly optimize for the desired pairwise fairness goals during training.…”