Introducing systems‐ and complexity‐informed evaluation

Abstract

For the last several decades and recently amidst the COVID‐19 pandemic, many in the global evaluation communities call for shifts from linear, reductionist ways of thinking and working to approaches that embrace systems and complexity. In this introductory chapter, we orient readers to key systems and complexity traditions and terms and how these have been put to use in the evaluation field. Doing so provides a foundation from which to engage with the subsequent chapters. We close this chapter with highlights from the case examples featured in this issue.

Designing Contribution Analysis of Participatory Programming to Tackle the Worst Forms of Child Labour

“…Rogers and Weiss 2007) and complexity-aware 3 (e.g. Gates et al 2021) evaluation designs are twofold. On the one hand, there are technical challenges in deciding an appropriate mix of methods that respond to specific evaluation questions to ensure credibility and confidence in conclusions that can be drawn.…”

Systems thinking concepts within a collaborative program evaluation methodology: The Hermes Program Evaluation

“…), questioning the nature of knowledge, our reflections on our knowledge processes, as well as assumptions about how knowledge can be justified. Since critical systems thinking regards systems as epistemological constructs (see Gates, 2018; Gates et al, 2021), questioning systems constructs is part of questioning epistemological assumptions. We will briefly present an example of this type of boundary critique practice by making reference to a particular evaluation (for more details, see Pinzon-Salcedo, 2002).…”

“…Examples of the use of systems thinking in evaluation can be found in several publications, such as a 2021 special issue of the journal “Evaluation” on Policy Evaluation for a Complex World from the Centre for the Evaluation of Complexity Across the Nexus (CECAN). Other examples can be found in a special issue of the New Directions for Evaluation journal, where Gates et al (2021) identify five main ways systems thinking and complexity ideas are being considered within the evaluation field. The interested reader can also consult other materials on evaluation and complexity science, such as the Systems Thinking in Evaluation book by Williams and Imam (2006) and Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation (Bicket et al, 2020), which presents implications of complexity theory for evaluation providing advice on the selection of evaluation complexity-appropriate approaches, discussing strengths and weaknesses of diverse methods in multiple circumstances.…”

“…However, none of these methods and concepts have been broadly applied in evaluation practice (Barbrook-Johnson et al, 2021). The use of complexity science, systems thinking and systems science in evaluation has been discussed for more than two decades, allowing some applications of its perspectives and tools in complex problematic situations faced by evaluators (Barbrook-Johnson et al, 2021; Gates et al, 2021). Barbrook-Johnson et al (2021) present the work of a multiplicity of pioneers’ in the use of systems and complexity science ideas and methods in the field of evaluation, which we cannot present here due to lack of space, but which we recommend to all interested readers.…”

Data pipeline for managing field experiments

“…These continuous technological advancements constantly challenge how scientific work is planned, conducted, and communicated [5] . Research projects are becoming more complex over time, accounting for a wider range of interactions, and requiring cross-discipline engagement [ 6 , 11 , 12 ]. In agricultural and environmental research, projects need to increasingly focus on multiple outcomes (e.g.…”