A discussion of current crime forecasting indices and an improvement to the prediction efficiency index for applications

Abstract

Hot-spot maps regularly aid many policing resource allocation decisions in today's data-driven age. However, it is unclear what forecasting algorithm(s) should be used to create these maps. To address this gap, we must be able to assess how "good" a generated hot-spot map is. Currently, four main metrics are used for evaluation: the prediction accuracy index (PAI), the recapture rate index (RRI), the prediction efficiency index (PEI), and the prediction efficiency index* (PEI*). This article discusses PAI, RRI, and PEI's strengths and weaknesses, articulates and justifies PEI*, and demonstrates the differences in calculations and interpretations of each metric. We argue that PEI* measures the efficiency of a crime forecasting algorithm while being operationally realistic and should be used in conjunction with other appropriate measures.Keywords (max 6): hot-spot forecasting indices, Prediction Efficiency Index, crime prediction, crime analysis, recapture rate index, predication accuracy index there must be standardized ways to evaluate and compare various hot-spot mapping models to ensure the effective and efficient use of limited police resources that also builds the body of evidence.There are various commonly used hot-spot mapping models to identify crime hot-spots, from naïve models to advanced machine learning algorithms. However, the literature is conflicting as to which model is the best; some argue the Random Forest (RF), Kernel-Density Estimation (KDE), Multilayer Perception, and Risk Terrain Modeling (RTM) are the most promising (Kounadi et al. 2020;Hart 2020).Due to a lack of consistent terminology, evaluation criteria, and reporting of initial parameters, it is unclear which models are best (Kounadi et al. 2020).This article summarizes the current knowledge of crime forecasting indices that were created to evaluate hot-spot mapping models. We refer the reader to White and Hunt (2022) for a summary of statistical measures used in developing hot-spot mapping models. We also further the crime index literature by formally articulating and justifying the prediction efficiency index* (PEI*) as a more applied, operationally realistic index to measure the efficiency of crime forecasting algorithms. Lastly, we provide a computational example and case study to demonstrate the differences in crime forecasting indices. Review of Crime Forecasting Indices Early Crime IndicesAs hot-spot mapping models developed, many researchers and practitioners used visual inspection of the generated hot-spot maps and other initial crime indices that were easy to implement. However, the simplicity of these initial indices resulted in inconsistency, subjectivity, and 'gaming.' For a summary of hot-spot map evaluation methods before 2000, we refer the reader to Jefferis (1999). By the early 2000s, researchers (and analysts) relied on two crime forecasting indices to compare the results of hot-spot mapping models, the "hit rate" (ratio of the percentage of crimes forecasted to the percentage of area forecasted) and the search efficie...