Representing theories of change: technical challenges with evaluation consequences

Davies, Rick

doi:10.1080/19439342.2018.1526202

Cited by 35 publications

(47 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Those that have addressed (and articulated themselves) the call in the literature for more complexity-appropriate, or systems-aware, ToC maps include Abercrombie et al (2018), Davies (2018) and Ling (2012). The former develops practical guidance on how to build ToC maps that can support efforts towards system change, using five rules-of-thumb which are elaborated in detail, these are understand context, know yourself (i.e.…”

Section: Introductionmentioning

confidence: 99%

“…how might we need to change), think systemically, learn and adapt, and recognise change is personal. Though framed as technical challenges, Davies (2018) systematically sets out a set of potential weaknesses in the practice of ToC, mostly related to incompleteness and vagueness. A set of six possible solutions to these issues are then laid out: more nuanced description of connections in ToCs, use of better software to draw ToCs, use of some basic network analysis, participatory development of ToCs, and two more technical analysis and modelling approaches to extend and complement ToCs (predictive analytics and dynamic modelling).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Participatory systems mapping for complex energy policy evaluation

Barbrook-Johnson

Penn

2021

Evaluation

View full text Add to dashboard Cite

The use of complexity science in evaluation has received growing attention over the last 20 years. We present the use of a novel complexity-appropriate method – Participatory Systems Mapping – in two real-world evaluation contexts and consider how this method can be applied more widely in evaluation. Participatory Systems Mapping involves the production of a causal map of a system by a diverse set of stakeholders. The map, once refined and validated, can be analysed and used in a variety of ways in an evaluation or in evaluation planning. The analysis approach combines network analysis with subjective information from stakeholders. We suggest Participatory Systems Mapping shows great potential to offer value to evaluators due to the unique insights it offers, the relative ease of its use, and its complementarity with existing evaluation approaches and methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Participatory systems mapping for complex energy policy evaluation

Barbrook-Johnson

Penn

2021

Evaluation

View full text Add to dashboard Cite

show abstract

“…Furthermore, diverse representations of ToCs have resulted in debate over their "best" manner of use (Davies, 2018;Prinsen & Nijhof, 2015). Whilst more nuanced and comprehensive ToCs are required to capture greater complexity within contemporary interventions (discussed below), the resultant elaborate and complicated diagrammatic representations may often prove counterproductive to broad multiactor engagement (discussed below), and the endgoal of influencing desired change (Davies, 2018;Valters, 2015). Therefore, designing a ToC requires a "balancing-act" of remaining simple, readable, and useable, for the purposes of communication and consensus, whilst still providing sufficient detail to reliably account for an intervention's real-world context (Davies, 2018;Koleros, Mulkerne, Oldenbeuving, & Stein, 2020;Mayne, 2015).…”

Section: Toc Terminology and Representationmentioning

confidence: 99%

Using Theory of Change to improve post‐2020 conservation: A proposed framework and recommendations for use

Rice

Bavinck

2020

Conservat Sci and Prac

View full text Add to dashboard Cite

Contemporary conservation must address social well-being while still protecting biodiversity. Accordingly, the objective of the Convention on Biological Diversity's recent Zero Draft Post-2020 Global Biodiversity Framework is to sustainably meet the needs of people while reducing biodiversity loss. However, frequent "failures" in achieving this social-ecological balance necessitates more holistic, systematic, and adaptive post-2020 conservation interventions. The Theory of Change (ToC) approach provides a useful and flexible tool to support this endeavor. However, debate persists over its usefulness, and "best" manner of use. This paper explores the elements of, and proposes a framework for developing robust conservation ToC pathways. The framework emphasizes the importance of producing a shared vision of desired results and actions, and associated causal assumptions, among actors. Furthermore, evaluation is considered key to informing required ongoing adaptation to better achieve desired results. The paper also critically explores the challenges associated with ToC, and makes recommendations for its improved use in post-2020 conservation. In particular, we aim to inform the implementation and mainstreaming of the Post-2020 Global Biodiversity Framework, especially at a national-and local-level. The framework and discussion should be relevant to a broad range of conservation actors at various scales that must address linked social and ecological objectives.

show abstract

“…Yet perhaps given their flexible nature, ToCs tend to be under-developed. For instance, it is not uncommon for ToCs to miss key steps such as outlining the assumptions, considering the context, or describing the nature of causal connections (Davies, 2018). And in some recently published cases, assumptions appeared to be left out entirely (Makowiecka et al, 2019;Sarma et al, 2019;Thompson & Moret, 2019).…”

Section: Partially Supported Evidencementioning

confidence: 99%

“…Furthermore, while many ToCs aim to examine the causal relationships among activities, outputs, and outcomes, they often proceed in a linear fashion, leading to some practitioners coming to view ToC as a "glorified logframe" (James, 2011, p. 10). The limited feedback loops also imply a process of change that has a linear trajectory (Davies, 2018). This linearity makes it difficult for ToCs to capture complexity and unexpected outcomes (Leeuw & Donaldson, 2015).…”

Section: Concern 7 Overlooking Complexitymentioning

confidence: 99%

Toward Learning from Change Pathways: Reviewing Theory of Change and Its Discontents

Lam

2020

Canadian Journal of Program Evaluation

View full text Add to dashboard Cite

The concept of Theory of Change (ToC) is well established in the evaluation literature, underpinning substantial research and practice efforts. However, its ability to facilitate learning has been increasingly critiqued. The objective of this paper is to identify, characterize, and evaluate concerns over the use of ToCs based on a review of relevant studies. Seven concerns are found: distinguishing ToCs from other evaluation approaches, conceptual vagueness, under-developed ToCs, contributing theoretical knowledge, stakeholder engagement, considering context, and capturing complexity. Priority areas for improvement include integrating context and complexity throughout the ToC process, contributing theoretical knowledge, and engaging stakeholders as appropriate.

show abstract

Representing theories of change: technical challenges with evaluation consequences

Cited by 35 publications

References 18 publications

Participatory systems mapping for complex energy policy evaluation

Participatory systems mapping for complex energy policy evaluation

Using Theory of Change to improve post‐2020 conservation: A proposed framework and recommendations for use

Toward Learning from Change Pathways: Reviewing Theory of Change and Its Discontents

Contact Info

Product

Resources

About