Complex systems for evaluation of public health interventions: a critical review

Egan, Matt; McGill, Elizabeth; Penney, Tarra L.; Meier, Petra; Savona, Natalie; Vocht, Frank de; Popay, Jennie; Cummins, Steven; Smith, Richard; White, Martin; Er, Vanessa; Petticrew, Mark

doi:10.1016/s0140-6736(18)32053-1

Cited by 21 publications

(33 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to move beyond this impasse, alternative research designs may be required to evaluate multifaceted approaches that function within a complex adaptive system (29,33,(44)(45)(46)(47). Signs of this are starting to emerge.…”

Section: Summary Of Resultsmentioning

confidence: 99%

A Secondary Analysis of the Childhood Obesity Prevention Literature through a Wider Determinants of Health Lens: Implications for Research Funders, Researchers, Policymakers and Practitioners

Nobles

Summerbell

Brown

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Over the last 25 years, we have made great advances in our understanding of the complex adaptive systems which drive childhood obesity. Within this, the important role of upstream interventions which focus on the wider determinants of health (WDoH) has been acknowledged. Concurrently, the number of intervention studies aiming to prevent childhood obesity has increased exponentially. One might anticipate that our understanding of theory has been embedded into the development and implementation of interventions in real time. Methods: We tested this hypothesis by conducting secondary analysis of the interventions included in the Cochrane Review on “Interventions for Preventing Obesity in Children”, published since 1993. All 153 randomised controlled trials (RCTs) were independently coded by two authors against the WDoH model using an adaptive framework synthesis approach. We used aspects of the Action Mapping Tool from Public Health England to facilitate our coding and to visualise our findings in contrast to 226 perceived causes of obesity. Results: We found that the proportion of interventions which targeted downstream (e.g. individual and family behaviours) as opposed to upstream (e.g. infrastructure, environmental, policy) determinants has not changed over time (from 1993 to 2015), with most intervention efforts (57.9%) aiming to change individual lifestyle factors via education-based approaches. Almost half of the interventions (45%) targeted two or more levels of the WDoH. Where interventions targeted some of the wider determinants, this was often achieved via upskilling teachers to deliver educational content to children. No notable difference in design or implementation was observed between interventions targeting children of varying ages (0-5 years, 6-12 years, 13-18 years). Conclusions: This study highlights that interventions, evaluated via RCTs, have persisted to focus on downstream, individualistic determinants of obesity over the last 25 years, despite the step change in our understanding of its complex aetiology. We hope that the findings from our analysis will challenge research funders, researchers, policymakers and practitioners to reflect upon, and critique, the evidence-based paradigm in which we operate, and call for a shift in focus of new evidence which takes a WDoH lens.

show abstract

Section: Summary Of Resultsmentioning

confidence: 99%

A Secondary Analysis of the Childhood Obesity Prevention Literature through a Wider Determinants of Health Lens: Implications for Research Funders, Researchers, Policymakers and Practitioners

Nobles

Summerbell

Brown

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…An evaluation study accompanying interventions in complex systems can serve multiple purposes depending on the phase of the programme [ 54 ]. At the start of the LIKE programme, our evaluation aimed to support the development and implementation of the actions by: developing an understanding of the system; framing and co-creating actions that can change the system, supporting sustainable implementation of the co-created actions and in assessing the potential value of the actions and overall system dynamics and participatory-based approach.…”

Section: Discussionmentioning

confidence: 99%

A System Dynamics and Participatory Action Research Approach to Promote Healthy Living and a Healthy Weight among 10–14-Year-Old Adolescents in Amsterdam: The LIKE Programme

Waterlander

Pinzon

Verhoeff

et al. 2020

IJERPH

View full text Add to dashboard Cite

This paper describes the design of the LIKE programme, which aims to tackle the complex problem of childhood overweight and obesity in 10–14-year-old adolescents using a systems dynamics and participatory approach. The LIKE programme focuses on the transition period from 10-years-old to teenager and was implemented in collaboration with the Amsterdam Healthy Weight Programme (AHWP) in Amsterdam-East, the Netherlands. The aim is to develop, implement and evaluate an integrated action programme at the levels of family, school, neighbourhood, health care and city. Following the principles of Participatory Action Research (PAR), we worked with our population and societal stakeholders as co-creators. Applying a system lens, we first obtained a dynamic picture of the pre-existing systems that shape adolescents’ behaviour relating to diet, physical activity, sleep and screen use. The subsequent action programme development was dynamic and adaptive, including quick actions focusing on system elements (quick evaluating, adapting and possibly catalysing further action) and more long-term actions focusing on system goals and/or paradigm change. The programme is supported by a developmental systems evaluation and the Intervention Level Framework, supplemented with routinely collected data on weight status and health behaviour change over a period of five years. In the coming years, we will report how this approach has worked to provide a robust understanding of the programme’s effectiveness within a complex dynamic system. In the meantime, we hope our study design serves as a source of inspiration for other public health intervention studies in complex systems.

show abstract

“…Specifically, our approach allows for a robust, systems-based assessment of community programs which account for complexity within different components of the system, among different actors, and at different levels. This approach provides an addition to the growing body of literature of systems approaches to the analyses of health programs [59][60][61][62][63][64]. While the concepts and tools that comprise this approach have been applied singularly in the assessment of health systems and programs, this is the first approach to our knowledge that unites their strengths cohesively.…”

Section: Discussionmentioning

confidence: 99%

A systems approach to assessing complexity in health interventions: an effectiveness decay model for integrated community case management

Karim

Muñoz

Mäusezahl

et al. 2020

Global Health Action

View full text Add to dashboard Cite

Complexity is inherent to any system or program. This is especially true of integrated interventions, such as integrated community case management (iCCM). iCCM is a child health strategy designed to provide services through community health workers (CHWs) within hard-to-reach areas of low-and-middle-income countries (LMICs). It is comprised of many interlinked program components, processes and stakeholders. Elucidating the complexity of such programs is essential to designing interventions that respond to local contexts and successfully plan for sustainable integration. A pragmatic approach has yet to be developed that holistically assesses the many dimensions of iCCM or other integrated programs, their alignment with local systems, and how well they provide effective care. We propose an accessible systems approach to both measuring systems effectiveness and assessing its underlying complexity using a combination of systems thinking tools. We propose an effectiveness decay model for iCCM implementation to measure where patient loss occurs along the trajectory of care. The approach uses process mapping to examine critical bottlenecks of iCCM processes, their influence on effectiveness decay, and their integration into local systems; regression analysis and structural equation modeling to determine effects of key indicators on programmatic outcomes; and qualitative analysis with causal loop diagramming to assess stakeholder dynamics and their interactions within the iCCM program. An accurate assessment of the quality, effectiveness, and strength of community-based interventions relies on more than measuring core indicators and program outcomes; it requires an exploration of how its actors and core components interact as part of a system. Our approach produces an interactive iCCM effectiveness decay model to understand patient loss in context, examines key systems issues, and uses a range of systems thinking tools to assess the dynamic interactions that coalesce to produce observed program outcomes.

show abstract

Complex systems for evaluation of public health interventions: a critical review

Cited by 21 publications

References 0 publications

A Secondary Analysis of the Childhood Obesity Prevention Literature through a Wider Determinants of Health Lens: Implications for Research Funders, Researchers, Policymakers and Practitioners

A Secondary Analysis of the Childhood Obesity Prevention Literature through a Wider Determinants of Health Lens: Implications for Research Funders, Researchers, Policymakers and Practitioners

A System Dynamics and Participatory Action Research Approach to Promote Healthy Living and a Healthy Weight among 10–14-Year-Old Adolescents in Amsterdam: The LIKE Programme

A systems approach to assessing complexity in health interventions: an effectiveness decay model for integrated community case management

Contact Info

Product

Resources

About