Overviews of reviews often have limited rigor: a systematic review

Pieper, Dawid; Buechter, Roland Brian; Jerinic, Petra; Eikermann, Michaela

doi:10.1016/j.jclinepi.2012.06.015

Cited by 164 publications

(200 citation statements)

References 27 publications

Supporting

Mentioning

196

Contrasting

Unclassified

Order By: Relevance

“…Thomson et al, [42] in contrast, advocated for OSR authors to ensure the evidence reviewed is up-to-date. Pieper et al [40] noted that 5% of the 126 overviews included in his systematic review of overview quality included a search for primary studies and stated that little empirical evidence is available to guide decisions about when and how to update reviews.…”

Section: Strengths and Limitationsmentioning

confidence: 99%

Preventing Child and Adolescent Anxiety Disorders: Overview of Systematic Reviews

et al. 2015

View full text Add to dashboard Cite

Overviews of systematic reviews (OSRs) provide rapid access to high quality, consolidated research evidence about prevention intervention options, supporting evidence-informed decision-making, and the identification of fruitful areas of new research. This OSR addressed three questions about prevention strategies for child and adolescent anxiety: (1) Does the intervention prevent anxiety diagnosis and/or reduce anxiety symptoms compared to passive controls? (2) Is the intervention equal to or more effective than active controls? (3) What is the evidence quality (EQ) for the intervention? Prespecified inclusion criteria identified systematic reviews and meta-analyses (2000-2014) with an AMSTAR quality score ≥ 3/5. EQ was rated using Oxford evidence levels EQ1 (highest) to EQ5 (lowest). Three reviews met inclusion criteria. One narrative systematic review concluded school-based interventions reduce anxiety symptoms. One meta-analysis pooled 65 randomized controlled trials (RCTs; any intervention) and reported a small, statistically significant reduction in anxiety symptoms and diagnosis incidence. Neither review provided pooled effect size estimates for specific intervention options defined by type (i.e., universal/selective/indicated), intervention content, or comparison group (i.e., passive/active control), thus precluding EQ ratings. One meta-analysis pooled trials of vigorous exercise and reported small, nonstatistically significant reductions in anxiety symptoms for comparisons against passive and active controls (EQ1). Better use of primary studies in meta-analyses, including program-specific pooled effect size estimates and network meta-analysis is needed to guide evidence-informed anxiety prevention program choices. RCTs of innovative community/primary care based interventions and web-based strategies can fill knowledge gaps.

show abstract

Section: Strengths and Limitationsmentioning

confidence: 99%

Preventing Child and Adolescent Anxiety Disorders: Overview of Systematic Reviews

et al. 2015

View full text Add to dashboard Cite

show abstract

“…As a result, current practice when conducting overviews is driven largely by personal experience and trial and error, and published overviews show considerable variation in their methods and reporting [4–6]. In recent years, a number of overview authors have recognized the methodological challenges inherent in conducting overviews and expressed a need for comprehensive, up-to-date guidance for overviews [4, 5, 13]. …”

Section: Introductionmentioning

confidence: 99%

What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary

et al. 2016

View full text Add to dashboard Cite

BackgroundOverviews of reviews (overviews) compile data from multiple systematic reviews to provide a single synthesis of relevant evidence for decision-making. Despite their increasing popularity, there is limited methodological guidance available for researchers wishing to conduct overviews. The objective of this scoping review is to identify and collate all published and unpublished documents containing guidance for conducting overviews examining the efficacy, effectiveness, and/or safety of healthcare interventions. Our aims were to provide a map of existing guidance documents; identify similarities, differences, and gaps in the guidance contained within these documents; and identify common challenges involved in conducting overviews.MethodsWe conducted an iterative and extensive search to ensure breadth and comprehensiveness of coverage. The search involved reference tracking, database and web searches (MEDLINE, EMBASE, DARE, Scopus, Cochrane Methods Studies Database, Google Scholar), handsearching of websites and conference proceedings, and contacting overview producers. Relevant guidance statements and challenges encountered were extracted, edited, grouped, abstracted, and presented using a qualitative metasummary approach.ResultsWe identified 52 guidance documents produced by 19 research groups. Relatively consistent guidance was available for the first stages of the overview process (deciding when and why to conduct an overview, specifying the scope, and searching for and including systematic reviews). In contrast, there was limited or conflicting guidance for the latter stages of the overview process (quality assessment of systematic reviews and their primary studies, collecting and analyzing data, and assessing quality of evidence), and many of the challenges identified were also related to these stages. An additional, overarching challenge identified was that overviews are limited by the methods, reporting, and coverage of their included systematic reviews.ConclusionsThis compilation of methodological guidance for conducting overviews of healthcare interventions will facilitate the production of future overviews and can help authors address key challenges they are likely to encounter. The results of this project have been used to identify areas where future methodological research is required to generate empirical evidence for overview methods. Additionally, these results have been used to update the chapter on overviews in the next edition of the Cochrane Handbook for Systematic Reviews of Interventions.Electronic supplementary materialThe online version of this article (doi:10.1186/s13643-016-0367-5) contains supplementary material, which is available to authorized users.

show abstract

“…The Overview Quality Assessment Questionnaire (OQAQ) [3, 4] and Assessment of Multiple Systematic Reviews (AMSTAR) [5–7] are two widely used tools for the assessment of systematic reviews. Two surveys of overviews (systematic reviews of reviews) found both instruments to be used frequently in this context [8, 9]. …”

Section: Introductionmentioning

confidence: 99%

Can AMSTAR also be applied to systematic reviews of non-randomized studies?

2014

Self Cite

View full text Add to dashboard Cite

BackgroundThere is a lack of an instrument to evaluate systematic reviews of non-randomized studies in epidemiological research. The Assessment of Multiple Systematic Reviews (AMSTAR) is widely used to evaluate the scientific quality of systematic reviews, but it has not been validated for SRs of non-randomized studies. The objective of this paper is to report our experience in applying AMSTAR to systematic reviews of non-randomized studies in terms of applicability, reliability and feasibility. Thus, we applied AMSTAR to a recently published review of 32 systematic reviews of non-randomized studies investigating the hospital volume-outcome relationship in surgery.ResultsThe inter-rater reliability was high (0.76), albeit items 8 (scientific quality used in formulating conclusions), 9 (appropriate method to combine studies), and 11 (conflicts of interest) scored moderate (≤0.58). However, there was a high heterogeneity between the two pairs of reviewers. In terms of feasibility, AMSTAR proved easy to apply to systematic reviews of non-randomized studies, each review taking 5–10 minutes to complete. We faced problems in applying three items, mainly related to scientific quality of the included studies.ConclusionsAMSTAR showed good psychometric properties, comparable to prior findings in systematic reviews of randomized controlled trials. AMSTAR can be applied to systematic reviews of non-randomized studies, although there are some item specific issues users should be aware of. Revisions and extensions of AMSTAR might be helpful.

show abstract

Overviews of reviews often have limited rigor: a systematic review

Cited by 164 publications

References 27 publications

Preventing Child and Adolescent Anxiety Disorders: Overview of Systematic Reviews

Preventing Child and Adolescent Anxiety Disorders: Overview of Systematic Reviews

What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary

Can AMSTAR also be applied to systematic reviews of non-randomized studies?

Contact Info

Product

Resources

About