Improving police decision making: general principles and practical applications of receiver operating characteristic analysis

Abstract

Receiver operating characteristic (ROC) analysis is a widely used and accepted method for improving decision making performance across a range of diagnostic settings. The goal of this paper is to demonstrate how ROC analysis can be used to improve the quality of decisions made routinely in a policing context. To begin, I discuss the general principles underlying the ROC approach and demonstrate how one can conduct the analysis. Several practical applications of ROC analysis are then presented by drawing on a number of policing tasks where the procedure has been used already (bite mark identification and linking serial crimes) or where it could be used in the future (statement validity assessment and determining the veracity of suicide notes). I conclude by considering briefly some of the potential difficulties that may be encountered when using ROC analysis in the policing context and offer some possible solutions to these problems.

Addressing problems with traditional crime linking methods using receiver operating characteristic analysis

“…We have previously argued that linkage analysis can be conceptualized as a signal detection problem, at least when the linking task involves the consideration of whether a pair of crimes has been committed by the same offender (Bennell, 2005; Bennell & Canter, 2002; Bennell & Jones, 2005). Indeed, there are many similarities between this linking task and other diagnostic decisions.…”

Linking serial residential burglary: comparing the utility of modus operandi behaviours, geographical proximity, and temporal proximity

“…It is important to note here that the statistical dependence between the linked and unlinked crime pairs violated the assumption of independence for logistic regression (Bennell & Canter, 2002). ROC analysis does not have such an assumption and has additional advantages over logistic regression when assessing predictive accuracy (Bennell, 2005). The results of the ROC analysis should therefore be given greater credence.…”

Postcharge mental health diversion: characteristics of clients and predictors of success

“…These analyses provide a specific probability of hits (a correct prediction of the outcome) and false alarms (an incorrect prediction of the outcome), allowing for the selection of the model with the highest predictive utility. The AUC statistic is particularly useful for evaluating predictive accuracy because it considers the entire ROC curve, rather than only one ROC point (Bennell, 2005). AUC's can range from 0 to 1, with higher numbers representing higher predictive accuracy, and AUC's at the midpoint (.5) indicating predictive accuracy at chance level.…”