Broader Issues Surrounding Model Transparency in Criminal Justice Risk Scoring

Rudin, Cynthia; Wang, Caroline; Coker, Beau

doi:10.1162/99608f92.038c43fe

Cited by 5 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another flourishing track to combat decision bias is to invent more transparent, delicate and de-biasing algorithms. It has been widely shown that enhanced algorithmic transparency and interpretability can help eliminate bias, and sometimes simple and transparent models are able to outperform complex black-box models [34,[50][51][52]. [57] demonstrate that Bayesian investment model can significantly improve investors' investment decisions based on other investment models.…”

Section: Relevant Literaturementioning

confidence: 99%

Uncovering the Source of Machine Bias

Hu¹,

Huang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

We develop a structural econometric model to capture the decision dynamics of human evaluators on an online micro-lending platform, and estimate the model parameters using a real-world dataset. We find two types of biases in gender, i.e. preference-based bias and belief-based bias, are present in human evaluators' decisions. Both types of biases are in favor of female applicants. Through counterfactual simulations, we quantify the effect of gender bias on loan granting outcomes and the welfare of the company and the borrowers. Our results imply that both the existence of the preference-based bias and that of the belief-based bias reduce the company's profits. When the preference-based bias is removed, the company earns more profits. When the belief-based bias is removed, the company's profits also increase. Both increases result from raising the approval probability for borrowers, especially male borrowers, who eventually pay back loans. For borrowers, the elimination of either bias decreases the gender gap of the true positive rates in the credit risk evaluation. We also train machine learning algorithms on both the real-world data and the data from the counterfactual simulations. We compare the decisions made by those algorithms to see how evaluators' biases are inherited by the algorithms and reflected in machine-based decisions. We find that machine learning algorithms can mitigate both the preferencebased bias and the belief-based bias. CCS CONCEPTS• Computing methodologies → Machine learning.

show abstract

Section: Relevant Literaturementioning

confidence: 99%

Uncovering the Source of Machine Bias

Hu¹,

Huang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Code is law: how COMPAS affects the way the judiciary handles the risk of recidivism

Engel,

Linhardt,

Schubert

2024

Artif Intell Law

View full text Add to dashboard Cite

Judges in multiple US states, such as New York, Pennsylvania, Wisconsin, California, and Florida, receive a prediction of defendants’ recidivism risk, generated by the COMPAS algorithm. If judges act on these predictions, they implicitly delegate normative decisions to proprietary software, even beyond the previously documented race and age biases. Using the ProPublica dataset, we demonstrate that COMPAS predictions favor jailing over release. COMPAS is biased against defendants. We show that this bias can largely be removed. Our proposed correction increases overall accuracy, and attenuates anti-black and anti-young bias. However, it also slightly increases the risk that defendants are released who commit a new crime before tried. We argue that this normative decision should not be buried in the code. The tradeoff between the interests of innocent defendants and of future victims should not only be made transparent. The algorithm should be changed such that the legislator and the courts do make this choice.

show abstract

Is Machine Learning Really Unsafe and Irresponsible in Social Sciences? Paradoxes and Reconsideration from Recidivism Prediction Tasks

Liu,

2024

Asian J Criminol

View full text Add to dashboard Cite

Broader Issues Surrounding Model Transparency in Criminal Justice Risk Scoring

Cited by 5 publications

References 0 publications

Uncovering the Source of Machine Bias

Uncovering the Source of Machine Bias

Code is law: how COMPAS affects the way the judiciary handles the risk of recidivism

Is Machine Learning Really Unsafe and Irresponsible in Social Sciences? Paradoxes and Reconsideration from Recidivism Prediction Tasks

Contact Info

Product

Resources

About