Methods used to meta-analyse results from interrupted time series studies: A methodological systematic review protocol

Korevaar, Elizabeth; Karahalios, Amalia; Forbes, Andrew; Turner, Simon; McDonald, Steve; Taljaard, Monica; Grimshaw, Jeremy; Cheng, Allen; Bero, Lisa; McKenzie, Joanne E.

doi:10.12688/f1000research.22226.3

Cited by 6 publications

(11 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The paper is structured as follows. In this section, we briefly describe the review methods, with further details available in the protocol [15] (deviations from our planned methods are presented in Appendix A). In Section 3 , we present the results, structured according to Fig.…”

Section: Review Methodsmentioning

confidence: 99%

“…The complete eligibility criteria are available in the protocol [15] . The inclusion criteria were: 1) a "review" that included at least two ITS which met the review authors' definition of an ITS design; and 2) included at least one meta-analysis of ITS.…”

Section: Eligibility Criteriamentioning

confidence: 99%

“…Our search strategy was informed by previous publications that have reviewed ITS studies [ 11 , 12 , 20 ], and may be found in the supplementary material of the protocol [15] . Several databases were searched to capture the broad range of disciplines that use the ITS design.…”

Section: Literature Searchmentioning

confidence: 99%

“…Abstracts and titles were screened independently and in duplicate by two members of the review team (E.K., A.K., S.L.T., A.B.F., and J.E.M.). Full-text articles of eligible abstracts were retrieved, sorted by most recent first and screened (independently and in duplicate by two members of the review team) until the target sample size of 100 reviews [15] was reached. Conflicts in screening decisions at both stages were resolved via discussion among the review team.…”

Section: Study Selectionmentioning

confidence: 99%

“…The specific analytical decisions will be dictated by a range of factors such as: whether raw time series data are available (as would be the case in a "primary study with multiple series," or if the data could be extracted from graphs or tables of the included ITS studies in a "review of studies" [6][7][8][9] ); the desired meta-analysis approach (oneor two-stage); and, whether there is a need to re-analyze the primary ITS studies. Re-analysis of ITS could be necessary when, for example, autocorrelation has not been accounted for in the original analysis (potentially leading to estimates of standard errors that are too small) [10][11][12][13] , when the model structure in the original analysis is inappropriate [14] , or when inconsistent effect measures have been reported across the studies [14][15][16] . These decisions can lead to different analytical pathways as depicted in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Methodological systematic review recommends improvements to conduct and reporting when meta-analyzing interrupted time series studies

Korevaar¹,

Karahalios²,

Turner³

et al. 2022

Journal of Clinical Epidemiology

Self Cite

View full text Add to dashboard Cite

Section: Review Methodsmentioning

confidence: 99%

Section: Eligibility Criteriamentioning

confidence: 99%

Section: Literature Searchmentioning

confidence: 99%

Section: Study Selectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Methodological systematic review recommends improvements to conduct and reporting when meta-analyzing interrupted time series studies

Korevaar¹,

Karahalios²,

Turner³

et al. 2022

Journal of Clinical Epidemiology

Self Cite

View full text Add to dashboard Cite

Evaluation of statistical methods used to meta‐analyse results from interrupted time series studies: A simulation study

Korevaar,

Turner,

Forbes

et al. 2023

Research Synthesis Methods

Self Cite

View full text Add to dashboard Cite

Interrupted time series (ITS) are often meta‐analysed to inform public health and policy decisions but examination of the statistical methods for ITS analysis and meta‐analysis in this context is limited. We simulated meta‐analyses of ITS studies with continuous outcome data, analysed the studies using segmented linear regression with two estimation methods [ordinary least squares (OLS) and restricted maximum likelihood (REML)], and meta‐analysed the immediate level‐ and slope‐change effect estimates using fixed‐effect and (multiple) random‐effects meta‐analysis methods. Simulation design parameters included varying series length; magnitude of lag‐1 autocorrelation; magnitude of level‐ and slope‐changes; number of included studies; and, effect size heterogeneity. All meta‐analysis methods yielded unbiased estimates of the interruption effects. All random effects meta‐analysis methods yielded coverage close to the nominal level, irrespective of the ITS analysis method used and other design parameters. However, heterogeneity was frequently overestimated in scenarios where the ITS study standard errors were underestimated, which occurred for short series or when the ITS analysis method did not appropriately account for autocorrelation. The performance of meta‐analysis methods depends on the design and analysis of the included ITS studies. Although all random effects methods performed well in terms of coverage, irrespective of the ITS analysis method, we recommend the use of effect estimates calculated from ITS methods that adjust for autocorrelation when possible. Doing so will likely to lead to more accurate estimates of the heterogeneity variance.

show abstract

Evaluation of statistical methods used to meta-analyse results from interrupted time series studies: a simulation study

Korevaar

Turner

Forbes

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Background Interrupted time series (ITS) are often meta-analysed to inform public health and policy decisions but examination of the statistical methods for ITS analysis and meta-analysis in this context is limited. Methods We simulated meta-analyses of ITS studies with continuous outcome data, analysed the studies using segmented linear regression with two estimation methods [ordinary least squares (OLS) and restricted maximum likelihood (REML)], and meta-analysed the immediate level- and slope-change effect estimates using fixed-effect and (multiple) random-effects meta-analysis methods. Simulation design parameters included varying series length; magnitude of lag-1 autocorrelation; magnitude of level- and slope-changes; number of included studies; and, effect size heterogeneity. Results All meta-analysis methods yielded unbiased estimates of the interruption effects. All random effects meta-analysis methods yielded coverage close to the nominal level, irrespective of the ITS analysis method used and other design parameters. However, heterogeneity was frequently overestimated in scenarios where the ITS study standard errors were underestimated, which occurred for short series or when the ITS analysis method did not appropriately account for autocorrelation. Conclusions The performance of meta-analysis methods depends on the design and analysis of the included ITS studies. Although all random effects methods performed well in terms of coverage, irrespective of the ITS analysis method, we recommend the use of effect estimates calculated from ITS methods that adjust for autocorrelation when possible. Doing so will likely to lead to more accurate estimates of the heterogeneity variance.

show abstract

Methods used to meta-analyse results from interrupted time series studies: A methodological systematic review protocol

Cited by 6 publications

References 20 publications

Methodological systematic review recommends improvements to conduct and reporting when meta-analyzing interrupted time series studies

Methodological systematic review recommends improvements to conduct and reporting when meta-analyzing interrupted time series studies

Evaluation of statistical methods used to meta‐analyse results from interrupted time series studies: A simulation study

Evaluation of statistical methods used to meta-analyse results from interrupted time series studies: a simulation study

Contact Info

Product

Resources

About