Which Method Predicts Recidivism Best?: A Comparison of Statistical, Machine Learning and Data Mining Predictive Models

Abstract

Summary. Using criminal population conviction histories of recent offenders, prediction mod els are developed that predict three types of criminal recidivism: general recidivism, violent recidivism and sexual recidivism. The research question is whether prediction techniques from modern statistics, data mining and machine learning provide an improvement in predictive performance over classical statistical methods, namely logistic regression and linear discrim inant analysis. These models are compared on a large selection of performance measures. Results indicate that classical methods do equally well as or better than their modern counterparts. The predictive performance of the various techniques differs only slightly for general and violent recidivism, whereas differences are larger for sexual recidivism. For the general and violent recidivism data we present the results of logistic regression and for sexual recidivism of linear discriminant analysis.

Isolating modeling effects in offender risk assessment

“…Two broad types of techniques have been of particular interest: decision trees (i.e., classification tree analysis, as well as random forests), and artificial neural networks (Berk et al 2009;Brodzinski et al 1994;Caulkins et al 1996;Gardner et al 1996;Grann and Langstrom 2007;Monahan et al 2000;Palocsay et al 2000;Silver et al 2000;Stalans et al 2004;Steadman et al 2000;Thomas et al 2005;Tollenaar and van der Heijden 2013). Although additional methods exist, our current interest is focused on examining those techniques that have, at the time of writing this piece, garnered the most attention-decision trees/random forests, and neural networks, and comparing them to the traditional logistic regression.…”

“…Specifically, we hypothesize that NN will perform better than RF, which, in turn, will perform better than logistic regression. Additionally, we make said comparisons of predictive validity while providing methodological advances from previous studies and focusing on increased criterion validity (Liu et al 2011;Tollenaar and van der Heijden 2013), specifically using one of the largest samples ever assembled to examine risk model prediction (N = 297,600), a fixed 3-year follow-up, previously validated predictor items with known strength of specificity and predictive power (Barnoski and Drake 2007), a heterogeneous sample (including females), and comparison including one broadband (felony) and three narrowband models (violent, drug, and sex reconvictions). Additionally, models were fine-tuned to prevent over-fitting, a common characteristic of data-mining procedures.…”

“…The forest leads to identifying an "average" tree, used for prediction purposes. The forest is first fitted on a training subset of the data, and its performance is tested on a validation subset (Liu et al 2011;Tollenaar and van der Heijden 2013).…”

StatRec - Performance, Validation and Preservability of a Static Risk Prediction Instrument

“…Jointly, these factors provided maximum predictive power. Using different alternatives for logistic regression models did not provide any improved performance (see Tollenaar and Van der Heijden, 2013). All parameters are statistically significant, which is not surprising given the large number of observations.…”

Predicting Recidivism of Juvenile Offenders

“…Another line of research has developed new statistical and econometric methods to analyze recidivism (Schmidt and Witte 1989;Rhodes 1989;Bunday and Kiri 1992;Pezzin 1995;Escarela, Francis, and Soothill 2000;Brame, Bushway, and Paternoster 2003;Koopman et al 2008;Berk et al 2009, Tollenaar andvan der Heijden 2013). Escarela, Francis, and Soothill (2000) developed an exponential mixture model for competing risks to analyze the risk of three different types of recidivism, in a model that allowed for the presence of desisters; their data covered a period of 32 years.…”