Learning Certifiably Optimal Rule Lists

Angelino, Elaine; Larus-Stone, Nicholas; Alabi, Daniel; Seltzer, Margo; Rudin, Cynthia

doi:10.1145/3097983.3098047

Cited by 147 publications

(238 citation statements)

References 43 publications

(51 reference statements)

Supporting

Mentioning

220

Contrasting

Unclassified

Order By: Relevance

“…In the criminal justice context, Jung et al (2017) show that "simple rules that consider only two features -age and prior FTAs [failureto-appear in court] -perform nearly identically to state-of-the-art machine learning models (random forest and lasso regression) that incorporate all 64 available features". 6 Other authors have shown that it is possible to construct simple two-feature prediction tools that perform as well as the well-known risk assessment tool COMPAS, which has access to 136 input variables, or a non-linear support vector machine trained on 7 input variables (Angelino et al, 2017;Dressel and Farid, 2018).…”

Section: Algorithmic Risk Assessment In Criminal Justicementioning

confidence: 99%

Algorithmic Risk Assessment in the Hands of Humans

Stevenson

Doleac

2019

SSRN Journal

View full text Add to dashboard Cite

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte.We evaluate the impacts of adopting algorithmic predictions of future offending (risk assessments) as an aid to judicial discretion in felony sentencing. We find that judges' decisions are influenced by the risk score, leading to longer sentences for defendants with higher scores and shorter sentences for those with lower scores. However, we find no robust evidence that this reshuffling led to a decline in recidivism, and, over time, judges appeared to use the risk scores less. Risk assessment's failure to reduce recidivism is at least partially explained by judicial discretion in its use. Judges systematically grant leniency to young defendants, despite their high risk of reoffending. This is in line with a long standing practice of treating youth as a mitigator in sentencing, due to lower perceived culpability.Such a conflict in goals may have led prior studies to overestimate the extent to which judges make prediction errors. Since one of the most important inputs to the risk score is effectively off-limits, risk assessment's expected benefits are curtailed. We find no evidence that risk assessment affected racial disparities statewide, although there was a relative increase in sentences for black defendants in courts that appeared to use risk assessment most. We conduct simulations to evaluate how race and age disparities would have changed if judges had fully complied with the sentencing recommendations associated with the algorithm. Racial disparities might have increased slightly, but the largest change would have been higher relative incarceration rates for defendants under the age of 23.In the context of contentious public discussions about algorithms, our results highlight the importance of thinking about how man and machine interact. JEL Classification:K4

show abstract

Section: Algorithmic Risk Assessment In Criminal Justicementioning

confidence: 99%

Algorithmic Risk Assessment in the Hands of Humans

Stevenson

Doleac

2019

SSRN Journal

View full text Add to dashboard Cite

show abstract

“…For a company like Northpointe to invest the time and effort into creating such a model, it seems reasonable to afford the company intellectual property protections. However, as we discussed, machine learning methods -either standard black-box or, better yet, recently-developed interpretable ones -can predict equally well or better than bespoke models like COMPAS (21)(22)(23)(24). For important applications like criminal justice, academics have always been willing to devote their time and energy.…”

Section: Introductionmentioning

confidence: 99%

“…Work in machine learning has shown that complicated, black-box, proprietary models are not necessary for recidivism risk assessment. Researchers have shown (on several datasets, including the data from Broward County) that interpretable models are just as accurate as black box machine learning models for predicting recidivism (21)(22)(23)(24)(25)(26). These simple models involve age and counts of past crimes, and indicate that younger people, and those with longer criminal histories, are more likely to reoffend.…”

Section: Introductionmentioning

confidence: 99%

The Age of Secrecy and Unfairness in Recidivism Prediction

Rudin¹,

Wang²,

Coker³

2020

Harvard Data Science Review

Self Cite

View full text Add to dashboard Cite

and ultimately stagnating because there is no clear definition of fairness and competing definitions are largely incompatible. We argue that the focus on the question of fairness is misplaced, as these algorithms fail to meet a more important and yet readily obtainable goal: transparency. As a result, creators of secret algorithms can provide incomplete or misleading descriptions about how their models work, and various other kinds of errors can easily go unnoticed.By trying to partially reconstruct the COMPAS model -a recidivism-risk scoring model used throughout the criminal justice system -we show that it does not seem to depend linearly on the defendant's age, despite statements to the contrary by the model's creator. This observation has not been made before despite many recently published papers on this algorithm. Furthermore, by subtracting from COMPAS its (hypothesized) nonlinear age component, we show that COMPAS does not necessarily depend on race, contradicting ProPublica's analysis, which assumed linearity in age. In other words, faulty assumptions about a proprietary algorithm lead to faulty conclusions that go unchecked. Were the algorithm transparent in the first place, this likely would

show abstract

“…A recent approach also using MDL and probabilistic rule lists (MRL) [4] is aimed at describing rather than classifying and cannot deal with multiclass problems or a large number of candidates. Interpretable decision sets (IDS) [25] and certifiable optimal rules (CORELS) [3] use similar rules but do not provide probabilistic models or predictions.…”

Section: Related Workmentioning

confidence: 99%

Interpretable multiclass classification by MDL-based rule lists

Proença

Leeuwen

2020

Information Sciences

View full text Add to dashboard Cite

Interpretable classifiers have recently witnessed an increase in attention from the data mining community because they are inherently easier to understand and explain than their more complex counterparts. Examples of interpretable classification models include decision trees, rule sets, and rule lists. Learning such models often involves optimizing hyperparameters, which typically requires substantial amounts of data and may result in relatively large models.In this paper, we consider the problem of learning compact yet accurate probabilistic rule lists for multiclass classification. Specifically, we propose a novel formalization based on probabilistic rule lists and the minimum description length (MDL) principle. This results in virtually parameter-free model selection that naturally allows to trade-off model complexity with goodness of fit, by which overfitting and the need for hyperparameter tuning are effectively avoided. Finally, we introduce the Classy algorithm, which greedily finds rule lists according to the proposed criterion.We empirically demonstrate that Classy selects small probabilistic rule lists that outperform state-of-the-art classifiers when it comes to the combination of predictive performance and interpretability. We show that Classy is insensitive to its only parameter, i.e., the candidate set, and that compression on the training set correlates with classification performance, validating our MDL-based selection criterion.

show abstract

Learning Certifiably Optimal Rule Lists

Cited by 147 publications

References 43 publications

Algorithmic Risk Assessment in the Hands of Humans

Algorithmic Risk Assessment in the Hands of Humans

The Age of Secrecy and Unfairness in Recidivism Prediction

Interpretable multiclass classification by MDL-based rule lists

Contact Info

Product

Resources

About