Considerations for Developing Predictive Spatial Models of Crime and New Methods for Measuring Their Accuracy

Abstract

A literature review of the important trends in predictive crime modeling and the existing measures of accuracy was undertaken. It highlighted the need for a robust, comprehensive and independent evaluation and the need to include complementary measures for a more complete assessment. We develop a new measure called the penalized predictive accuracy index (PPAI), propose the use of the expected utility function to combine multiple measures and the use of the average logarithmic score, which measures accuracy differently than existing measures. The measures are illustrated using hypothetical examples. We illustrate how PPAI could identify the best model for a given problem, as well as how the expected utility measure can be used to combine different measures in a way that is the most appropriate for the problem at hand. It is important to develop measures that empower the practitioner with the ability to input the choices and preferences that are most appropriate for the problem at hand and to combine multiple measures. The measures proposed here go some way towards providing this ability. Further development along these lines is needed.

A discussion of current crime forecasting indices and an improvement to the prediction efficiency index for applications

“…Relevant statistical metrics are summarized in White and Hunt (2022). Crime analysts should avoid solely using PAI, as it is not a reliable metric (Joshi et al 2021;Galvis, Hernández-Romero, and Gómez 2022). Instead, we suggest analysts use PEI and/or PEI* as outlined here and look for better metrics as they are developed.…”

“…where measured PAI refers to calculating PAI over a single time period, and predictive PAI refers to evaluating PAI from two different time indices (Van Patten, McKledin-Coner, and Cox 2009). To address various study areas, one study developed the penalized predictive accuracy index (PPAI), where the area ratio is penalized (Joshi et al 2021). However, mathematicians have shown that PAI and its adaptions are still susceptible to artificial manipulation since PAI increases as subspace size (e.g., grid cell height, street length) decreases (Galvis, Hernández-Romero, and Gómez 2022).…”