A Comparison of Statistical Methods for Identifying Out-of-Date Systematic Reviews

Abstract

BackgroundSystematic reviews (SRs) can provide accurate and reliable evidence, typically about the effectiveness of health interventions. Evidence is dynamic, and if SRs are out-of-date this information may not be useful; it may even be harmful. This study aimed to compare five statistical methods to identify out-of-date SRs.MethodsA retrospective cohort of SRs registered in the Cochrane Pregnancy and Childbirth Group (CPCG), published between 2008 and 2010, were considered for inclusion. For each eligible CPCG review, data were extracted and “3-years previous” meta-analyses were assessed for the need to update, given the data from the most recent 3 years. Each of the five statistical methods was used, with random effects analyses throughout the study.ResultsEighty reviews were included in this study; most were in the area of induction of labour. The numbers of reviews identified as being out-of-date using the Ottawa, recursive cumulative meta-analysis (CMA), and Barrowman methods were 34, 7, and 7 respectively. No reviews were identified as being out-of-date using the simulation-based power method, or the CMA for sufficiency and stability method. The overall agreement among the three discriminating statistical methods was slight (Kappa = 0.14; 95% CI 0.05 to 0.23). The recursive cumulative meta-analysis, Ottawa, and Barrowman methods were practical according to the study criteria.ConclusionOur study shows that three practical statistical methods could be applied to examine the need to update SRs.

Best Practice Guidelines and Essential Methodological Steps to Conduct Rigorous and Systematic Meta‐Reviews

“…The datedness of the literature in the meta‐review is important to consider, because as Pattanittum and colleagues argued, “Evidence is dynamic, and if systematic reviews are out‐of‐date this information may not only be unhelpful, it may be harmful” (Pattanittum, Laopaiboon, Moher, Lumbiganon, & Ngamjarus, , p. 1). With meta‐reviews specifically, datedness is important at multiple levels.…”

“…Alternatively, some quantitative methods can assist with this task (e.g. see Pattanittum et al., ). Meta‐review teams can also scrape the dates of included studies from extant meta‐analyses to produce a temporal model that predicts how many studies may have appeared following the most recent review's search.…”

A shared latent space matrix factorisation method for recommending new trial evidence for systematic review updates

“…The approach is similar to ours in that it requires knowing which studies were included in existing systematic reviews. However, the approach appears to have used between 6,502…”

“…Performing systematic reviews is time and resource intensive and even determining when a systematic review needs updating often requires completion of the searching and screening steps of the systematic review process. To facilitate this assessment, a number of tools and guidelines have been developed that aim to identify when new relevant research becomes available, or estimate the risk that the results of the systematic review may have substantially changed due to new evidence [3][4][5][6][7][8][9][10].…”

Verankerung interprofessioneller Teamarbeit in den Qualitätsmanagement-Systemen der medizinischen Rehabilitation – Eine Bestandsaufnahme

“…Eine Möglichkeit wäre durch Nachrecherchen nach Primärpublikationen diesem Prob lembereich zu begegnen, was in unserer Untersuchung jedoch nicht erfolgte. Zu dieser Thematik liegen methodische Arbeiten vor [53,54]. Da in die vorliegende Untersuchung relativ wenige Reviews (N = 6) eingingen, besteht durch das methodische Vor gehen bei der Auswahl der Determinanten allgemein die Gefahr der verzerrten Gewichtung der Determinanten.…”