Evaluation of public health interventions from a complex systems perspective: A research methods review

McGill, Elizabeth; Er, Vanessa; Penney, Tarra L.; Egan, Matt; White, Martin; Meier, Petra; Whitehead, Margaret; Lock, Karen; Cuevas, Rachel Anderson de; Smith, Richard; Savona, Natalie; Rutter, Harry; Marks, Dalya; Vocht, Frank de; Cummins, Steven; Popay, Jennie; Petticrew, Mark

doi:10.1016/j.socscimed.2021.113697

Cited by 124 publications

(129 citation statements)

References 109 publications

Supporting

Mentioning

112

Contrasting

Unclassified

Order By: Relevance

“…There is a growing call for more robust methods of measuring their benefit and impact. These findings further support conclusions by McGill and colleagues that methodological innovations in the evaluation of systems approaches in healthcare are needed in the increasingly complex health sector (McGill et al, 2021). Despite these limitations, we argue that what is needed is not a quick dismissal of these approaches, especially given the potential demonstrated in the studies identified in the referenced systematic review and the clear policy support for improving how we tackle health system challenges.…”

Section: Discussionsupporting

confidence: 85%

Systems Approaches to Healthcare Systems Design and Care Delivery: An Overview of the Literature

2021

View full text Add to dashboard Cite

The healthcare sector is facing significant challenges that require a systems approach, resulting in a rapid growth in the application of systems approaches in healthcare since the beginning of the 21st century. Consequently, healthcare practitioners and policymakers now desire to understand the evidence-base for the approach, but little evidence of the kind desired exists. This paper is a first step in conducting a narrative review of the application of systems approaches in healthcare based on a systematic review of the academic and grey literature. First, the emergence of the approach in healthcare is explored. Second, specific examples of applications of systems approaches in healthcare are examined to identify any missing elements in current practice. Third, fourteen reviews of the approach in healthcare published in the last ten years are analysed. The results suggest that the use of the approach in healthcare will most likely continue to increase, however, significant work remains for the design and systems community to demonstrate the effectiveness of systems approaches, specifically in providing convincing measures of impact on patient and service outcomes.

show abstract

Section: Discussionsupporting

confidence: 85%

Systems Approaches to Healthcare Systems Design and Care Delivery: An Overview of the Literature

2021

View full text Add to dashboard Cite

show abstract

“…McGill et al, for example, conducted a review of methods and found 74 unique studies that described evaluations of complex systems. The authors concluded that “a range of complex systems methods can be utilized, adapted, or combined to produce different types of evaluative evidence” ( 3 ). They went on to note that methodological innovation in systems evaluation may be of particular value in generating “stronger evidence” ( 3 ).…”

Section: Introductionmentioning

confidence: 99%

One Health Evaluation of Antimicrobial Use and Resistance Surveillance: A Novel Tool for Evaluating Integrated, One Health Antimicrobial Resistance and Antimicrobial Use Surveillance Programs

Haworth‐Brockman

Saxinger

Miazga-Rodriguez

et al. 2021

Front. Public Health

View full text Add to dashboard Cite

We describe the development, application and utility of our novel, One Health Evaluation of Antimicrobial Use and Resistance Surveillance (OHE-AMURS) tool that we created to evaluate progress toward integrated, One Health surveillance of antimicrobial resistance (AMR) and antimicrobial use (AMU) as a complex system in Canada. We conducted a qualitative inquiry into the current state of policy and programs for integrated AMR/AMU surveillance using explicit and tacit knowledge. To assess the “messy” state of public health surveillance program development, we synthesized recommendations from previous reports by the National Collaborating Centre for Infectious Diseases and the Canadian Council of Chief Veterinary Officers; conducted an environmental scan to find all federal, provincial, and territorial AMR/AMU surveillance programs in Canada; and conducted semi-structured interviews with Canadian subject matter experts. To integrate evidence from these different sources we adapted two published tools to create a new evaluation matrix, deriving 36 components of the ideal integrated AMR/AMU surveillance system. Our two-way matrix tool allowed us to examine seven common, foundational elements of sustainable programs for each component, and assign a stage of development/sustainability ranking for each component according to the matrix definitions. Our adaptable novel tool allowed for granular and repeatable assessment of the many components of a complex surveillance system. The assessment proved robust and exacting to ensure transparency in our methods and results. The matrix allows flexible assignment of program components based on program principles, and stages can be adapted to evaluate any aspect of an AMR/AMU surveillance or other multi-faceted, multi-jurisdictional system. Future refinement should include an assessment of the scope of surveillance components.

show abstract

“…This paper has explored the impact of Dry January on January drinking outcomes, but it is possible that the campaign (and other similar campaigns) may have had a broader systems effect by changing culture and affecting long-term and overall consumption, which we have not been able to evaluate here. Future research might explore how these initiatives have impacted alcohol reduction activities more broadly via increased acceptability of abstinence or might take a complex systems perspective ( McGill et al, 2021 , 2020 ) to explore how the broader system in which campaigns such as Dry January take place may interact with any changes driven by such campaigns.…”

Section: Discussionmentioning

confidence: 99%

“…Future evaluations might take a broader systems perspective (e.g. ( McGill et al, 2021 , 2020 ) to evaluate the Dry January campaign in the context of system-level determinants of alcohol consumption.…”

Section: Discussionmentioning

confidence: 99%

Has the increased participation in the national campaign ‘Dry January’ been associated with cutting down alcohol consumption in England?

Case

Angus

Vocht

et al. 2021

Drug and Alcohol Dependence

Self Cite

View full text Add to dashboard Cite

show abstract

Evaluation of public health interventions from a complex systems perspective: A research methods review

Cited by 124 publications

References 109 publications

Systems Approaches to Healthcare Systems Design and Care Delivery: An Overview of the Literature

Systems Approaches to Healthcare Systems Design and Care Delivery: An Overview of the Literature

One Health Evaluation of Antimicrobial Use and Resistance Surveillance: A Novel Tool for Evaluating Integrated, One Health Antimicrobial Resistance and Antimicrobial Use Surveillance Programs

Has the increased participation in the national campaign ‘Dry January’ been associated with cutting down alcohol consumption in England?

Contact Info

Product

Resources

About