Characterizing Fairness Over the Set of Good Models Under Selective Labels

Abstract

Algorithmic risk assessments are increasingly used to make and inform decisions in a wide variety of high-stakes settings. In practice, there is often a multitude of predictive models that deliver similar overall performance, an empirical phenomenon commonly known as the "Rashomon Effect." While many competing models may perform similarly overall, they may have different properties over various subgroups, and therefore have drastically different predictive fairness properties. In this paper, we develop a framework for characterizing predictive fairness properties over the set of models that deliver similar overall performance, or "the set of good models." We provide tractable algorithms to compute the range of attainable group-level predictive disparities and the disparity minimizing model over the set of good models. We extend our framework to address the empirically relevant challenge of selectively labelled data in the setting where the selection decision and outcome are unconfounded given the observed data features. We illustrate our methods in two empirical applications. In a real world credit-scoring task, we build a model with lower predictive disparities than the benchmark model, and demonstrate the benefits of properly accounting for the selective labels problem. In a recidivism risk prediction task, we audit an existing risk score, and find that it generates larger predictive disparities than any model in the set of good models.

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges

“…The knowledge of the functions that lie within Rashomon sets sees value in multiple interesting use cases. Models with various important properties besides interpretability can exist in the Rashomon set, including fairness (Coston et al, 2021) and monotonicity. Thus, understanding the properties of the Rashomon set could be pivotal for analysis of a complex machine learning problem and its possible modeling choices.…”

The Impact of Algorithmic Risk Assessments on Human Predictions and its Analysis via Crowdsourcing Studies

“…In the 'decreasing' setting, participants were first shown the offenders for whom the RAI had made accurate predictions and then those for whom the RAI's predictions were inaccurate. 17 See Table 2 for the distribution of participants across conditions. Here we only focus on the comparison of the controlled and decreasing conditions.…”

“…16 Note that only the metric computed in (i) follows the definition of influence presented in §3.5. 17 The randomization scheme differed across the two dimensions of variation due to the aforementioned technical issue rather than for methodological reasons. We first collected an initial batch of surveys with only the random setting.…”

“…While it can be challenging to assess the predictive performance and fairness properties of RAIs in isolation [17,33,46,47,51], investigating their impact on systems where the human is the ultimate decision maker introduces a new layer of complexity. Perhaps the most direct evidence bearing on these questions comes from longitudinal studies of real-world systems [16,20], particularly within the criminal justice setting.…”

How Costly is Noise? Data and Disparities in Consumer Credit

“…See alsoArnold et al (2020a) drawing the analogy between selective labels and unobserved potential outcomes, andRambachan and Roth (2019) for a discussion of how selection into training data can have subtle effects on the predictive power of algorithms trained on those data.18 This is a type of reject inference by extrapolation. (See alsoCoston et al (2021) for an application of this type of reject inference to credit data.) Unlike most applications of this type of reject inference, we leverage that we observe other applicant credit behavior after the application regardless of the outcome of the application.19 We conduct robustness tests using an all-encompassing any-default variable provided by TransUnion that includes all loan products reported to the credit bureau.…”