Methodological systematic review recommends improvements to conduct and reporting when meta-analyzing interrupted time series studies

Korevaar, Elizabeth; Karahalios, Amalia; Turner, Simon; Forbes, Andrew Benjamin; Taljaard, Monica; Cheng, Allen; Grimshaw, Jeremy; Bero, Lisa; McKenzie, Joanne E.

doi:10.1016/j.jclinepi.2022.01.010

Cited by 15 publications

(73 citation statements)

References 29 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ordinary least squares (OLS) estimators, commonly used in practice 8,12 , can be used to estimate the regression parameters and their standard errors 21 . A key assumption of OLS is that the model errors are uncorrelated between observations, which may be violated with time series data.…”

Section: Estimation Methods For Its Analysismentioning

confidence: 99%

“…The method chosen for inclusion in this study for its wide use in practice 8,34,43 , the Waldtype normal distribution (WT) confidence interval 48 , is calculated as:…”

Section: Confidence Interval Calculationmentioning

confidence: 99%

“…A quasi-experimental non-randomised design that is often used to evaluate the impact of interruptions targeted at populations is the interrupted time series (ITS) design [3][4][5] . This design is immune to common threats to internal validity compared with other nonrandomised designs (e.g., uncontrolled before-after design), and as such, is often included in systematic reviews [6][7][8] . The results from multiple ITS studies within systematic reviews may be statistically combined using meta-analysis methods; the findings of which underpin review conclusions 9,10 .…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Evaluation of statistical methods used to meta-analyse results from interrupted time series studies: a simulation study

Korevaar

Turner

Forbes

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Background Interrupted time series (ITS) are often meta-analysed to inform public health and policy decisions but examination of the statistical methods for ITS analysis and meta-analysis in this context is limited. Methods We simulated meta-analyses of ITS studies with continuous outcome data, analysed the studies using segmented linear regression with two estimation methods [ordinary least squares (OLS) and restricted maximum likelihood (REML)], and meta-analysed the immediate level- and slope-change effect estimates using fixed-effect and (multiple) random-effects meta-analysis methods. Simulation design parameters included varying series length; magnitude of lag-1 autocorrelation; magnitude of level- and slope-changes; number of included studies; and, effect size heterogeneity. Results All meta-analysis methods yielded unbiased estimates of the interruption effects. All random effects meta-analysis methods yielded coverage close to the nominal level, irrespective of the ITS analysis method used and other design parameters. However, heterogeneity was frequently overestimated in scenarios where the ITS study standard errors were underestimated, which occurred for short series or when the ITS analysis method did not appropriately account for autocorrelation. Conclusions The performance of meta-analysis methods depends on the design and analysis of the included ITS studies. Although all random effects methods performed well in terms of coverage, irrespective of the ITS analysis method, we recommend the use of effect estimates calculated from ITS methods that adjust for autocorrelation when possible. Doing so will likely to lead to more accurate estimates of the heterogeneity variance.

show abstract

Section: Estimation Methods For Its Analysismentioning

confidence: 99%

“…The method chosen for inclusion in this study for its wide use in practice 8,34,43 , the Waldtype normal distribution (WT) confidence interval 48 , is calculated as:…”

Section: Confidence Interval Calculationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of statistical methods used to meta-analyse results from interrupted time series studies: a simulation study

Korevaar

Turner

Forbes

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A methods review examining the characteristics of reviews that include ITS (14), found that such reviews are undertaken in a range of disciplines and topics, including public health, crime, economics, war and psychology. Therefore, the findings from this research are likely to have impact across disciplines.…”

Section: What Is Newmentioning

confidence: 99%

“…immediate level change) across the studies; the use of statistical methods that appropriately account for correlation arising from time series data (known as autocorrelation); and, complete reporting of the study effect estimate along with a measure of precision (e.g. confidence interval, standard error)(14, 15). When these criteria are not met for a particular ITS study, it will likely be excluded from the meta-analysis, resulting in a loss of information.…”

Section: Introductionmentioning

confidence: 99%

Effect estimates can be accurately calculated with data digitally extracted from interrupted time series graphs

Turner

Korevaar

Cumpston

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Background Interrupted Time Series (ITS) studies are frequently used to examine the impact of population-level interventions or exposures. Systematic reviews with meta-analyses including ITS designs may inform public health and policy decision-making. Re-analysis of ITS may be required for inclusion in meta-analysis. While publications of ITS rarely provide raw data for re-analysis, graphs are often included, from which time series data can be digitally extracted. However, the accuracy of effect estimates calculated from data digitally extracted from ITS graphs is currently unknown. Methods Forty-three ITS with available datasets and time series graphs were included. Time series data from each graph was extracted by four researchers using digital data extraction software. Data extraction errors were analysed. Segmented linear regression models were fitted to the extracted and provided datasets, from which estimates of immediate level and slope change (and associated statistics) were calculated and compared across the datasets. Results Although there were some data extraction errors of time points, primarily due to complications in the original graphs, they did not translate into important differences in estimates of interruption effects (and associated statistics). Conclusions Using digital data extraction to obtain data from ITS graphs should be considered in reviews including ITS. Including these studies in meta-analyses, even with slight inaccuracy, is likely to outweigh the loss of information from non-inclusion.

show abstract

Interventions for increasing energy efficiency in hospitals

Chen-Xu,

Kislaya,

Fernandes

et al. 2024

Cochrane Database of Systematic Reviews

View full text Add to dashboard Cite

Methodological systematic review recommends improvements to conduct and reporting when meta-analyzing interrupted time series studies

Cited by 15 publications

References 29 publications

Evaluation of statistical methods used to meta-analyse results from interrupted time series studies: a simulation study

Evaluation of statistical methods used to meta-analyse results from interrupted time series studies: a simulation study

Effect estimates can be accurately calculated with data digitally extracted from interrupted time series graphs

Interventions for increasing energy efficiency in hospitals

Contact Info

Product

Resources

About