Meeting the Challenges of Evidence-Based Policy: The Campbell Collaboration

Abstract

Evidence-based policy has much to recommend it, but it also faces significant challenges. These challenges reside not only in the dilemmas faced by policy makers but also in the quality of the evaluation evidence. Some of these problems are most effectively addressed by rigorous syntheses of the literature known as systematic reviews. Other problems remain, including the range of quality in systematic reviews and their general failure to be updated in light of new evidence or disseminated beyond the research community. Based on the precedent established in health care by the international Cochrane Collaboration, the newly formed Campbell Collaboration will prepare, maintain, and make accessible systematic reviews of research on the effects of social and educational interventions. Through mechanisms such as rigorous quality control, electronic publication, and worldwide coverage of the literature, the Campbell Collaboration seeks to meet challenges posed by evidence-based policy. Downloaded from 15 DONALD Campbell (1969) was an influential psychologist who wrote persuasively about the need for governments to take evaluation evidence into account in decisions about social programs. He also recognized, however, the limitations of the evidence-based approach and the fact that government officials would be faced with a number of political dilemmas that confined their use of research. The limits of evidence-based policy and practice, however, reside not only in the political pressures faced by decision makers when implementing laws and administrative directives or determining budgets; they also reside in problems with the research evidence.Questions such as, What works to reduce crime in communities? are not easily answered. The studies that bear on these questions are often scattered across different fields and written in different languages, are sometimes disseminated in obscure or inaccessible outlets, and can be of such questionable quality that interpretation is risky at best. How can policy and practice be informed, if not persuaded, by such a fragmented knowledge base comprising evaluative studies that range in quality? Which study, or set of studies, if any at all, ought to be used to influence policy? What methods ought to be used to appraise and analyze a set of separate studies bearing on the same question? And how can the findings be disseminated in such a way that the very people Donald Campbell cared about-the decision makers in government and elsewhere-receive findings from these analyses that they trust were not the product of advocacy? Donald Campbell unfortunately did not live long enough to bear witness to the creation of the international collaboration named in his honor that ambitiously attempts to address some of the challenges posed by evidence-based policy. The Campbell Collaboration was created to prepare, update, and disseminate systematic reviews of evidence on what works relevant to social and educational intervention (see http:// campbell.gse.upenn.edu). The target audience will include de...

Police crackdowns on illegal gun carrying: a systematic review of their impact on gun crime

“…This paper is drawn from a review undertaken for the Crime and Justice Group of the Campbell Collaboration, an international nonprofit organization that coordinates rigorous assessments of research evidence on the effectiveness of various interventions for social problems (Farrington and Petrosino 2001;Petrosino et al 2001). Campbell reviews are conducted using systematic methods.…”

“…Campbell reviews are intended to provide the most rigorous assessment possible of the effectiveness of studied interventions (Farrington and Petrosino 2001;Petrosino et al 2001). Accordingly, they generally focus on randomized experiments or nonrandomized quasi-experiments, particularly those involving a control group (Campbell and Stanley 1966).…”

Why do evaluation researchers in crime and justice choose non-experimental methods?

“…Since a number of highly influential reports in the 1970s and 1980s indicating a grim future for many criminal justice programs and policies (see Kelling et al 1974;Lipton et al 1975;Spelman and Brown 1984), the push towards evaluating criminal justice interventions to find effective treatments has defined the role of a number of researchers. This emphasis on program effectiveness can be seen in systematic reviews of programs (most notably, Sherman et al's 1997 report to congress), the increased use of meta-analyses to draw more parsimonious conclusions from the plethora of evaluation research (see, e.g., Andrews et al 1990;Cox et al 1995;Dowden et al 2003;Lipsey and Wilson 1993;Logan and Gaes 1993;Lösel and Koferl 1989;Prendergast et al 2000;Wilson 2000Wilson , 2001Wilson et al 2000Wilson et al , 2001Whitehead and Lab 1989), and the establishment of the Campbell Collaboration, 2 an organization which advocates for higher quality research and evidence-based policy (see Farrington and Petrosino 2001;Petrosino et al 2001).…”

Information retrieval in systematic reviews: a case study of the crime prevention literature

“…Chiefly, this included the internationally renowned Campbell Collaboration (Petrosino et al 2001). We drew heavily from their comprehensive guidance document on searching for studies by Hammerstrøm and colleagues (2010), and viewed their specialist SPECTR databases and YouTube channel of conference sessions.…”