Introducing EMMIE: an evidence rating scale to encourage mixed-method crime prevention synthesis reviews

Abstract

Objectives This paper describes the need for, and the development of, a coding system to distil the quality and coverage of systematic reviews of the evidence relating to crime prevention interventions. The starting point for the coding system concerns the evidence needs of policymakers and practitioners. Methods The proposed coding scheme (EMMIE) builds on previous scales that have been developed to assess the probity, coverage and utility of evidence both in health and criminal justice. It also draws on the principles of realist synthesis and review. Results The proposed EMMIE scale identifies five dimensions to which systematic reviews intended to inform crime prevention should speak. These are the Effect of intervention, the identification of the causal Mechanism(s) through which interventions are intended to work, the factors that Moderate their impact, the articulation of practical Implementation issues, and the Economic costs of intervention. Conclusions Systematic reviews of crime prevention, and the primary studies on which they are based, typically address the question of effect size, but are often silent on the other dimensions of EMMIE. This lacuna of knowledge is unhelpful to practitioners who want to know more than what might work to reduce crime. The EMMIE framework is intended to encourage the collection of primary data regarding these issues and the synthesis of such knowledge in future systematic reviews.

Wildlife crime: a conceptual integration, literature review, and methodological critique

“…Unfortunately, many of the key principles to conducting a proper evaluation such as a thorough understanding of: (1) context, the setting in which an intervention is implemented; (2) mechanisms, how the planned intervention is meant to work; and (3) outcomes, the practical effects produced by causal mechanisms that have been triggered are largely absent from the conservation research that has focused on crime. While it is difficult to operationalize these three concepts within an impact evaluation, Johnson et al (2015) recently developed mnemonic, EMMIE, seeks to organize the various evidential components required for decision makers to optimize the selection of interventions (see also Sidebottom and Bullock 2017). As outlined by Johnson et al (2015), the EMMIE framework refers to: E the overall effect direction and size (alongside major unintended effects) of an intervention and the confidence that should be placed on that estimate M the mechanisms/mediators activated by the policy, practice or program in question M the moderators/contexts relevant to the production/non-production of intended and major unintended effects of different sizes I the key sources of success and failure in implementing the policy, practice or program E the economic costs (and benefits) associated with the policy, practice or program (p. 463).…”

Right method, right price: the economic value and associated risks of experimentation

“…The acronym EMMIE spells out these main dimensions. Full details on the definition of these concepts and discussion on the detail of EMMIE can be found in Johnson et al (2015).…”

Gating Alleys to Reduce Crime: A Meta-Analysis and Realist Synthesis

“…Using the EMMIE framework (Johnson et al, 2015) to structure our review, we collated information on the effectiveness of alley gates, the causal mechanisms through which they are expected to work, the conditions necessary to activate such mechanisms, the implementation of alley gates and the economic costs and benefits of doing so. The results from our meta-analysis suggest that alley gating has been an effective burglary reduction measure, albeit with modest effect sizes.…”

“…This is the first objective of this review. Our second objective, informed by EMMIE (an acronym denoting Effect, Mechanism, Moderators, Implementation, and Economics, see Johnson, Tilley, & Bowers, 2015), is to identify the causal mechanisms through which alley gates are expected to reduce crime and the conditions under which they have been found to be effective, ineffective and/ or to produce unintended negative effects. To this aim we undertake a largely qualitative review drawing on the principles of realist evaluation and considering a wider range of alley gating studies.…”

“…EMMIE was devised as an acronym to encapsulate the types of evidence that studies might provide to inform decisionmaking concerning the funding and/or implementation of crime prevention schemes, in addition to and including information on intervention effectiveness (Johnson et al, 2015).…”