Simulation-based power and sample size calculation for designing interrupted time series analyses of count outcomes in evaluation of health policy interventions

Abstract

ObjectiveThe purpose of this study was to present the design, model, and data analysis of an interrupted time series (ITS) model applied to evaluate the impact of health policy, systems, or environmental interventions using count outcomes. Simulation methods were used to conduct power and sample size calculations for these studies.MethodsWe proposed the models and analyses of ITS designs for count outcomes using the Strengthening Translational Research in Diverse Enrollment (STRIDE) study as an example. The models we used were observation-driven models, which bundle a lagged term on the conditional mean of the outcome for a time series of count outcomes.ResultsA simulation-based approach with ready-to-use computer programs was developed to calculate the sample size and power of two types of ITS models, Poisson and negative binomial, for count outcomes. Simulations were conducted to estimate the power of segmented autoregressive (AR) error models when autocorrelation ranged from −0.9 to 0.9, with various effect sizes. The power to detect the same magnitude of parameters varied largely, depending on the testing level change, the trend change, or both. The relationships between power and sample size and the values of the parameters were different between the two models.ConclusionThis article provides a convenient tool to allow investigators to generate sample sizes that will ensure sufficient statistical power when the ITS study design of count outcomes is implemented.

Environmental Factors and Hyperacute Stroke Care Activity During the COVID-19 Pandemic: An Interrupted Time-Series Analysis

“…In addition, it has been shown previously that power increases if the numbers of data points are equally distributed before and after the intervention. 33 Given our limited time-frame between and post intervention, a larger number of data points prior may have not been beneficial. As the analyses were based on routine data collection, it is worth noting that despite our confidence in the data collection there may be potential for unobserved changes to data collection given the strain on resources, that could have affected our results.…”

Containment of Antibiotic REsistance—measures to improve antibiotic use in pregnancy, childbirth and young children (CAREChild): a protocol of a prospective, quasiexperimental interventional study in Lao PDR

“…Interrupted time series analysis will be used to compare the trend in the proportion of the primary outcome of interest the 12 months before the planned intervention and 12 months after it. 32 Moreover, in order to compare knowledge and attitudes of mothers and HCPs (pre–post intervention) we will use multilevel mixed effects generalised linear models (Stata command meglm) with different family distributions according to the outcome distribution (Stata option family). Results will be presented as effect estimates with 95% CIs and will primarily be done both together and separately for each healthcare facility.…”

Evaluating a Longitudinal Cohort of Clinics Engaging in the Family Planning Elevated Contraceptive Access Program: Study Protocol for a Comparative Interrupted Time Series Analysis

“…Primary outcomes of the program will be assessed using comparative (intervention vs control), multiperiod (before, during, and after), interrupted time series analyses (ITSAs) [ 36 , 37 ]. ITSAs have increasingly been demonstrated to be reliable assessments of community interventions and implementation research, providing rigorous outcome assessments in circumstances where randomized controlled trials are infeasible [ 38 , 39 ].…”