Policy evaluation for a complex world: Practical methods and reflections from the UK Centre for the Evaluation of Complexity across the Nexus

Abstract

The value of complexity science and related approaches in policy evaluation have been widely discussed over the last 20 years, not least in this journal. We are now at a crossroads; this Special Issue argues that the use of complexity science in evaluation could deepen and broaden rendering evaluations more practical and rigorous. The risk is that the drive to better evaluate policies from a complexity perspective could falter. This special issue is the culmination of 4 years’ work at this crossroads in the UK Centre for the Evaluation of Complexity Across the Nexus. It includes two papers which consider the cultural and organisational operating context for the use of complexity in evaluation and four methodological papers on developments and applications. Together, with a strong input from practitioners, these papers aim to make complexity actionable and expand the use of complexity ideas in evaluation and policy practice.

A summative content analysis of how programmes to improve the right to sexual and reproductive health address power

“…Programmes that seek to alter power relations might incorporate real-time learning and adaptation, and encounter uncertain trajectories of change, including pushback. Time frame and methodological limitations may undermine the effectiveness and assessment of a programme, and undercut effective scale-up or adaption and replication as the influence of social and political context is not elucidated 138 139. In brief, both the programmes implemented and the methods used to assess their impact likely do not correspond to the multilevel determinants of structure and agency.…”

The understanding, application and influence of complexity in national physical activity policy-making

“…In this paper, complexity is discussed in broad terms and encompasses theories, methods and academic disciplines associated with complexity science and systems-thinking [ 16 , 17 ]. Complexity science typically focuses on analysing change in dynamic systems over time, while systems-thinking is more concerned with the boundaries drawn around system structures and the interaction of agents (e.g.…”

“…The concept of complexity has gained traction in social sciences and policy research over the past quarter-century, with developments in research methods, methodological perspectives (e.g. implications for realism as applied in this paper), and how complexity can inform existing research practices [ 16 ]. Recently, there has been a further turn towards critical reflections on how these ideas are used in practice [ 16 , 19 – 21 ].…”

“…implications for realism as applied in this paper), and how complexity can inform existing research practices [ 16 ]. Recently, there has been a further turn towards critical reflections on how these ideas are used in practice [ 16 , 19 – 21 ]. It is generally accepted that complexity focuses on the behaviours and interactions within and between systems, which are largely unpredictable.…”

Conclusion: Transforming Evaluation Practice for “Business Unusual”

“…It is a radically diff erent perspective from a simplistic and linear reasoning approach in which micro-actions cause changes at macro levels. For more than two decades this has been discussed in the evaluation literature, but there are seemingly still too few examples of its application in practice being reported, though there are some signs of change (Barbrook-Johnson et al, 2021). Certainly, in part, this may well be due to a lack of evaluator expertise.…”