Making better use of natural experimental evaluation in population health

Craig, Peter; Campbell, Mhairi; Bauman, Adrian; Deidda, Manuela; Dundas, Ruth; Fitzgerald, Niamh; Green, John; Katikireddi, Srinivasa Vittal; Lewsey, Jim; Ogilvie, David; Vocht, Frank de; White, Martin

doi:10.1136/bmj-2022-070872

Cited by 19 publications

(14 citation statements)

References 39 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, the trade-off between bias and precision means that estimates are not necessarily more credible than those from any other design ( 42 ). As well as other randomized controlled designs, the importance of alternative evaluation designs is recognized in the recently updated UK Medical Research Council guidance for complex interventions which explicitly recognizes the need to consider a wider range of designs than just randomized controlled trials ( 35 ), including natural experiment designs ( 65 ).…”

Section: Possible Solutionsmentioning

confidence: 99%

Rethinking children’s physical activity interventions at school: A new context-specific approach

Jago

Salway

House

et al. 2023

Front. Public Health

Self Cite

View full text Add to dashboard Cite

Physical activity is important for children’s health. However, evidence suggests that many children and adults do not meet international physical activity recommendations. Current school-based interventions have had limited effect on physical activity and alternative approaches are needed. Context, which includes school setting, ethos, staff, and sociodemographic factors, is a key and largely ignored contributing factor to school-based physical activity intervention effectiveness, impacting in several interacting ways.ConceptualizationCurrent programs focus on tightly-constructed content that ignores the context in which the program will be delivered, thereby limiting effectiveness. We propose a move away from uniform interventions that maximize internal validity toward a flexible approach that enables schools to tailor content to their specific context.Evaluation designsEvaluation of context-specific interventions should explicitly consider context. This is challenging in cluster randomized controlled trial designs. Thus, alternative designs such as natural experiment and stepped-wedge designs warrant further consideration.Primary outcomeA collective focus on average minutes of moderate-to-vigorous intensity physical activity may not always be the most appropriate choice. A wider range of outcomes may improve children’s physical activity and health in the long-term. In this paper, we argue that greater consideration of school context is key in the design and analysis of school-based physical activity interventions and may help overcome existing limitations in the design of effective interventions and thus progress the field. While this focus on context-specific interventions and evaluation is untested, we hope to stimulate debate of the key issues to improve future physical activity intervention development and implementation.

show abstract

Section: Possible Solutionsmentioning

confidence: 99%

Rethinking children’s physical activity interventions at school: A new context-specific approach

Jago

Salway

House

et al. 2023

Front. Public Health

Self Cite

View full text Add to dashboard Cite

show abstract

“…The challenges in using RCTs to evaluate complex interventions to overcome social determinants of health are well described, 8 , 9 and most strategic decisions—particularly in Public Health—are made without the benefit of evidence from an RCT. So, what else constitutes acceptable evidence?…”

Section: Introductionmentioning

confidence: 99%

“…Otherwise, there is a risk of a blurring of intended target populations, outcomes, and analytical approaches. 9 , 56 Alongside the robust approaches to assess causal inference we have described, a published a priori protocol adds validity to the findings and has the potential to broaden the acceptability of NES as admissible evidence for causation. For reference, a detailed framework of what to include in the protocol has been recently published.…”

Section: Introductionmentioning

confidence: 99%

The role of natural experiments in hepatology research: filling the gap between clinical trials and service evaluations

Buchanan

Smith

Rowe

2023

Hepatology Communications

View full text Add to dashboard Cite

Research developing and testing interventions that address the social determinants of liver disease are urgently needed; however, this cannot be achieved using conventional clinical research designs. A different approach is needed to conduct widely applicable, inclusive, and community-based research that addresses upstream factors driving liver morbidity. Natural experimental studies encompass a well-established field of research methodology that is less familiar to clinical hepatologists than conventional research methods such as the randomized control trial. The key strength of natural experiments is that, when robustly designed, they can be used to imply causality from routinely collected data. As such, they are well placed to test the impact of community interventions that aim to address social determinants of liver disease that cannot feasibly be assessed in a randomized control trial. In this review, we define natural experiments and their potential utility. We then work through examples of where they have already been used in clinical hepatology to highlight a range of research designs, analytical approaches, and best practices regarding their conduct and reporting. In doing so, we hope to equip clinical hepatologists with another tool to ensure the hepatology community can meet the global liver disease epidemic with evidence-based interventions.

show abstract

“…Going forward, alcohol marketing restrictions should be accompanied by comprehensive evaluation programmes, making best use of naturalistic experimental designs that are well established as an appropriate way of evaluating the health impacts of policies, programmes and interventions [10]. These programmes should be guided by theories of change and logic models, involving structured identification of the key outcomes anticipated from the restrictions, the indicators needed to examine them (including differences among subpopulations) and the data required to ensure robust evaluation.…”

mentioning

confidence: 99%

Commentary on Manthey et al.: No more missed opportunities—We need to address the absence of robust and comprehensive evaluations about the real‐world impact of statutory restrictions on alcohol marketing

Critchlow

2024

Addiction

View full text Add to dashboard Cite

Making better use of natural experimental evaluation in population health

Cited by 19 publications

References 39 publications

Rethinking children’s physical activity interventions at school: A new context-specific approach

Rethinking children’s physical activity interventions at school: A new context-specific approach

The role of natural experiments in hepatology research: filling the gap between clinical trials and service evaluations

Commentary on Manthey et al.: No more missed opportunities—We need to address the absence of robust and comprehensive evaluations about the real‐world impact of statutory restrictions on alcohol marketing

Contact Info

Product

Resources

About