When is a Program Ready for Rigorous Impact Evaluation? The Role of a Falsifiable Logic Model

Epstein, Diana; Klerman, Jacob Alex

doi:10.1177/0193841x12474275

Cited by 39 publications

(48 citation statements)

References 17 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the experimental method has difficulty in taking into account spill-over effects from pilot-intervention areas to other areas, which makes the distinction between ''treated'' and ''control'' groups difficult (Ravallion 2010). Additionally, the relevance of context (Epstein and Klerman 2013) and the impossibility of establishing random and real control groups (Bamberger et al 2010) in cross-sector partnerships make the technique less appropriate to many partnership evaluation problems. As a result, quasi-experimental methods and more qualitative indicators are introduced in impact assessment research in which contextual variables are included .…”

Section: Methodological and Measurement Challengesmentioning

confidence: 99%

“…There exists the danger of taking credit for results that the partners cannot achieve (Ebrahim and Rangan 2013). In general, the pressure on organizations to measure performance and establish ''what works'' also in more complex areas like social programs, has increased (Epstein and Klerman 2013;Khagram and Thomas 2010;White 2009). Therefore, there is a greater emphasis on the consequences of partnerships (Biermann et al 2007) or impact instead of the more traditional focus on inputs and output effects.…”

Section: Organizational Pressurementioning

confidence: 99%

See 1 more Smart Citation

Enhancing the Impact of Cross-Sector Partnerships

et al. 2015

View full text Add to dashboard Cite

This paper addresses the topic of this special symposium issue: how to enhance the impact of crosssector partnerships. The paper takes stock of two related discussions: the discourse in cross-sector partnership research on how to assess impact and the discourse in impact assessment research on how to deal with more complex organizations and projects. We argue that there is growing need and recognition for cross-fertilization between the two areas. Cross-sector partnerships are reaching a paradigmatic status in society, but both research and practice need more thorough evidence of their impacts and of the conditions under which these impacts can be enhanced. This paper develops a framework that should enable a constructive interchange between the two research areas, while also framing existing research into more precise categories that can lead to knowledge accumulation.

show abstract

Section: Methodological and Measurement Challengesmentioning

confidence: 99%

Section: Organizational Pressurementioning

confidence: 99%

Enhancing the Impact of Cross-Sector Partnerships

et al. 2015

View full text Add to dashboard Cite

show abstract

“…In fact, we can easily generalize by ruling out: no program, no Leviton and Trujillo 5 effect. Epstein and Klerman (2013) have made the same point in the context of falsifiable logic models: Programs that do not have the necessary core components are a priori not effective and not worth evaluating.…”

Section: The Logic Of Inductionmentioning

confidence: 99%

Interaction of Theory and Practice to Assess External Validity

Leviton

Trujillo

2016

Eval Rev

View full text Add to dashboard Cite

In systematic programs of inquiry, as opposed to individual studies, the problems of context can be better addressed. Evaluators have advocated an interaction of theory and practice for many years, but the process can be made more systematic and useful. Systematic interaction can set priorities for assessment of external validity by examining the prevalence and importance of context features and treatment variations. Practitioner interaction with researchers and developers can assist in sharpening program theory, reducing uncertainty about treatment variations that are consistent or inconsistent with the theory, inductively ruling out the ones that are harmful or irrelevant, and helping set priorities for more rigorous study of context and treatment variation.

show abstract

“…There is risk associated with implementing strategies in a widespread manner that have not yet been subjected to rigorous evaluation – risk for inefficient use of resources and iatrogenic effects. Tension arises when a health problem is at epidemic status (e.g., prescription drug overdose), the number of identified effective strategies is low, and STLT agencies require guidance given consequences of inaction (e.g., declining health of a population); yet, history suggests that null results are often found when strategies are rigorously evaluated or evaluated at scale (Epstein & Klerman, 2012). When systematic reviews can only point to a small number of strategies, but the public health burden is high, it is necessary to prioritize the strategies with the most rigorous data supporting them, but also offer supplemental guidance to address the problem.…”

Section: The Use and Limitations Of Systematic Reviewsmentioning

confidence: 99%

Technical Packages in Injury and Violence Prevention to Move Evidence Into Practice

et al. 2016

View full text Add to dashboard Cite

Injury and violence prevention strategies have greater potential for impact when they are based on scientific evidence. Systematic reviews of the scientific evidence can contribute key information about which policies and programs might have the greatest impact when implemented. However, systematic reviews have limitations, such as lack of implementation guidance and contextual information, that can limit the application of knowledge. “Technical packages,” developed by knowledge brokers such as the federal government, nonprofit agencies, and academic institutions, have the potential to be an efficient mechanism for making information from systematic reviews actionable. Technical packages provide information about specific evidence-based prevention strategies, along with the estimated costs and impacts, and include accompanying implementation and evaluation guidance to facilitate adoption, implementation, and performance measurement. We describe how systematic reviews can inform the development of technical packages for practitioners, provide examples of technical packages in injury and violence prevention, and explain how enhancing review methods and reporting could facilitate the use and applicability of scientific evidence.

show abstract

When is a Program Ready for Rigorous Impact Evaluation? The Role of a Falsifiable Logic Model

Cited by 39 publications

References 17 publications

Enhancing the Impact of Cross-Sector Partnerships

Enhancing the Impact of Cross-Sector Partnerships

Interaction of Theory and Practice to Assess External Validity

Technical Packages in Injury and Violence Prevention to Move Evidence Into Practice

Contact Info

Product

Resources

About