Causal Inference Challenges and New Directions for Epidemiologic Research on the Health Effects of Social Policies

Abstract

Purposeof Review Epidemiologic research on the health effects of social policies is growing rapidly because of the potentially large impact of these policies on population health and health equity. We describe key methodological challenges faced in this nascent field and promising tools to enhance the validity of future studies. Recent Findings In epidemiologic studies of social policies, causal identification is most commonly pursued through confounder-control but use of instrument-based approaches is increasing. Researchers face challenges measuring relevant policy exposures; addressing confounding and positivity violations arising from co-occurring policies and time-varying confounders; deriving precise effect estimates; and quantifying and accounting for interference. Promising tools to address these challenges can enhance both internal validity (randomization, front door criterion for causal identification, new estimators that address interference and practical positivity violations) and external validity (data-driven methods for evaluating heterogeneous treatment effects; methods for transporting and generalizing effect estimates to new populations). Summary Common threats to validity in epidemiologic research play out in distinctive ways in research on the health effects of social policies. This is an active area of methodologic development, with ongoing advances to support causal inferences and produce policy-relevant findings. Researchers must navigate the tension between research questions of greatest interest and research questions that can be answered most accurately and precisely with the data at hand. Additional work is needed to facilitate integration of modern epidemiologic methods with econometric tools for policy evaluation and to increase the size and measurement quality of datasets.

A quantitative assessment of the frequency and magnitude of heterogeneous treatment effects in studies of the health effects of social policies

“…Second, HTEs are not routinely assessed along all potentially relevant dimensions ( Cintron et al, 2022 ). This study thus adds to accumulating evidence that substantial HTEs are common and should be routinely and systematically reported when evaluating the health effects of social policies ( Cintron et al, 2022 ; Matthay & Glymour, 2022 ; Petticrew et al, 2012 ). It also highlights the importance of study planning to ensure evaluations of social policies are sufficiently precise to evaluate HTEs.…”

“…Social policies may have substantial impacts on a broad range of population health outcomes, and a growing body of health research seeks to quantify their causal effects ( Matthay & Glymour, 2022 ). However, less research has evaluated differences in the effects of social policies across population subgroups.…”

“…Assessing heterogeneous treatment effects (HTEs) of social policies is critical to understand the implications of these policies for health inequities, and epidemiology researchers have increasingly called for HTE assessments for this reason ( Matthay & Glymour, 2022 ). The health effects of a policy on the population overall need not be in the same direction as the effects on inequities: policies that improve average health may exacerbate inequities, or conversely, policies that harm health on average may nonetheless narrow inequities.…”

Making better use of natural experimental evaluation in population health

“…Despite growing acceptance of the value of natural experimental evalutions,17181920 most discussion of their design focuses on quantitative methods 171821. Other aspects of study design, conduct, and interpretation, such as how to identify a good opportunity for a natural experimental evaluation, how open science principles such as registering a study should be applied, or how to place effect estimates into a broader framework of whether, how, and in what circumstances the intervention achieves its effects, have been largely neglected.…”

Changes in Racial/Ethnic Disparities in Opioid-Related Outcomes in Urban Areas during the COVID-19 Pandemic: A Rapid Review of the Literature

“…Examining the effect of these policies across racial/ethnic minoritized groups can inform efforts to reduce disparities and improve health among people with opioid use disorders. The use of causal methods for these evaluations can enhance their usefulness [ 44 ], but care must be taken when evaluating policies that co-occur with other policies (e.g., shelter at home concurrent with closures of certain outpatient locations) [ 45 ]. In addition, researchers should aim for consistency in reporting race and ethnicity in scientific manuscripts [ 46 ].…”