Systematic review found AMSTAR, but not R(evised)-AMSTAR, to have good measurement properties

Pieper, Dawid; Buechter, Roland Brian; Li, Lun; Prediger, Barbara; Eikermann, Michaela

doi:10.1016/j.jclinepi.2014.12.009

Cited by 137 publications

(106 citation statements)

References 38 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…In addition, we found that the AMSTAR assessments obtained for the SRs in our sample of seven overviews fell within the range of scores observed in a broader sample of overviews (all relevant overviews identified by [4] and [5] and contained within issue 12, 2016 of the CDSR). Our inter-rater reliability assessments were also similar to published data on agreement for AMSTAR [50]. Thus, the results of the current study, and subsequent recommendations for quality assessment of SRs in overviews, may generalize to a range of overviews examining healthcare interventions.…”

Section: Discussionsupporting

confidence: 85%

“…Thus, they concluded that the overall score is meaningful [21]. However, overall quality scores assume that all questions are equal (which can be difficult to justify) [50, 51], summing individual items may artificially increase the precision of the assessment, and studies have shown that incorporating overall quality of primary studies into meta-analyses can alter effect estimates in SRs [52, 53]. Despite the uncertainty regarding use of summary scores, there is a precedent for calculating and reporting overall AMSTAR assessments both in empirical studies assessing measurement properties of AMSTAR [20, 21, 29–32, 54–56] and in overviews of healthcare interventions [16, 18, 35, 36, 57–71], and incorporating overall quality of SRs into results of overviews has not been found to alter overview results [72].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions

Pollock

Fernandes

Hartling

2017

BMC Med Res Methodol

View full text Add to dashboard Cite

BackgroundOverviews of reviews (overviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. It is recommended that authors assess and report the methodological quality of SRs in overviews—for example, using A MeaSurement Tool to Assess systematic Reviews (AMSTAR). Currently, there is variation in whether and how overview authors assess and report SR quality, and limited guidance is available. Our objectives were to: examine methodological considerations involved in using AMSTAR to assess the quality of Cochrane and non-Cochrane SRs in overviews of healthcare interventions; identify challenges (and develop potential decision rules) when using AMSTAR in overviews; and examine the potential impact of considering methodological quality when making inclusion decisions in overviews.MethodsWe selected seven overviews of healthcare interventions and included all SRs meeting each overview’s inclusion criteria. For each SR, two reviewers independently conducted AMSTAR assessments with consensus and discussed challenges encountered. We also examined the correlation between AMSTAR assessments and SR results/conclusions.ResultsNinety-five SRs were included (30 Cochrane, 65 non-Cochrane). Mean AMSTAR assessments (9.6/11 vs. 5.5/11; p < 0.001) and inter-rater reliability (AC1 statistic: 0.84 vs. 0.69; “almost perfect” vs. “substantial” using the Landis & Koch criteria) were higher for Cochrane compared to non-Cochrane SRs. Four challenges were identified when applying AMSTAR in overviews: the scope of the SRs and overviews often differed; SRs examining similar topics sometimes made different methodological decisions; reporting of non-Cochrane SRs was sometimes poor; and some non-Cochrane SRs included other SRs as well as primary studies. Decision rules were developed to address each challenge. We found no evidence that AMSTAR assessments were correlated with SR results/conclusions.ConclusionsResults indicate that the AMSTAR tool can be used successfully in overviews that include Cochrane and non-Cochrane SRs, though decision rules may be useful to circumvent common challenges. Findings support existing recommendations that quality assessments of SRs in overviews be conducted independently, in duplicate, with a process for consensus. Results also suggest that using methodological quality to guide inclusion decisions (e.g., to exclude poorly conducted and reported SRs) may not introduce bias into the overview process.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-017-0325-5) contains supplementary material, which is available to authorized users.

show abstract

Section: Discussionsupporting

confidence: 85%

Section: Discussionmentioning

confidence: 99%

Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions

Pollock

Fernandes

Hartling

2017

BMC Med Res Methodol

View full text Add to dashboard Cite

show abstract

“…We used the “assessment of multiple systematic reviews” (AMSTAR) [19] which includes 11 items to judge the quality of each systematic review (Additional file 4). AMSTAR was found to be a reliable and valid measurement tool to assess the methodological quality of systematic reviews [20, 21], and it seems that all items can generally be applied to systematic reviews of non-randomized studies [22]. We added a supplemental question on reporting of dealing with multiple comparisons in primary studies.…”

Section: Methodsmentioning

confidence: 99%

Relationship between surgeon volume and outcomes: a systematic review of systematic reviews

2016

Self Cite

View full text Add to dashboard Cite

BackgroundThe surgeon volume-outcome relationship has been discussed for many years and its existence or nonexistence is of importance for various reasons. A lot of empirical work has been published on it. We aimed to summarize systematic reviews in order to present current evidence.MethodsMedline, Embase, Cochrane database of systematic reviews (CDSR), and health technology assessment websites were searched up to October 2015 for systematic reviews on the surgeon volume-outcome relationship. Reviews were critically appraised, and results were extracted and synthesized by type of surgical procedure/condition.ResultsThirty-two reviews reporting on 15 surgical procedures/conditions were included. Methodological quality of included systematic reviews assessed with the assessment of multiple systematic reviews (AMSTAR) was generally moderate to high albeit included literature partly neglected considering methodological issues specific to volume-outcome relationship. Most reviews tend to support the presence of a surgeon volume-outcome relationship. This is most clear-cut in colorectal cancer, bariatric surgery, and breast cancer where reviews of high quality show large effects.ConclusionsWhen taking into account its limitations, this overview can serve as an informational basis for decision makers. Our results seem to support a positive volume-outcome relationship for most procedures/conditions. However, forthcoming reviews should pay more attention to methodology specific to volume-outcome relationship. Due to the lack of information, any numerical recommendations for minimum volume thresholds are not possible. Further research is needed for this issue.Electronic supplementary materialThe online version of this article (doi:10.1186/s13643-016-0376-4) contains supplementary material, which is available to authorized users.

show abstract

“…27 The R(evised)-AMSTAR will not be used in our study, given its limited application and unknown measurement properties. 28 However, some items of AMSTAR may not be applicable to all the included systematic reviews. For instance, the item 9 'were the methods used to combine the findings of studies appropriate' (because not all the systematic reviews used a pooled estimate) is not relevant to some included studies, thereby being omitted from the quality evaluation.…”

Section: Quality Assessment Of Included Studiesmentioning

confidence: 99%

State of reporting of primary biomedical research: a scoping review protocol

Mbuagbaw

Samaan

et al. 2017

BMJ Open

View full text Add to dashboard Cite

IntroductionIncomplete or inconsistent reporting remains a major concern in the biomedical literature. Incomplete or inconsistent reporting may yield the published findings unreliable, irreproducible or sometimes misleading. In this study based on evidence from systematic reviews and surveys that have evaluated the reporting issues in primary biomedical studies, we aim to conduct a scoping review with focuses on (1) the state-of-the-art extent of adherence to the emerging reporting guidelines in primary biomedical research, (2) the inconsistency between protocols or registrations and full reports and (3) the disagreement between abstracts and full-text articles.Methods and analysesWe will use a comprehensive search strategy to retrieve all available and eligible systematic reviews and surveys in the literature. We will search the following electronic databases: Web of Science, Excerpta Medica Database (EMBASE), MEDLINE and Cumulative Index to Nursing and Allied Health Literature (CINAHL). Our outcomes are levels of adherence to reporting guidelines, levels of consistency between protocols or registrations and full reports and the agreement between abstracts and full reports, all of which will be expressed as percentages, quality scores or categorised rating (such as high, medium and low). No pooled analyses will be performed quantitatively given the heterogeneity of the included systematic reviews and surveys. Likewise, factors associated with improved completeness and consistency of reporting will be summarised qualitatively. The quality of the included systematic reviews will be evaluated using AMSTAR (a measurement tool to assess systematic reviews).Ethics and disseminationAll findings will be published in peer-reviewed journals and relevant conferences. These results may advance our understanding of the extent of incomplete and inconsistent reporting, factors related to improved completeness and consistency of reporting and potential recommendations for various stakeholders in the biomedical community.

show abstract

Systematic review found AMSTAR, but not R(evised)-AMSTAR, to have good measurement properties

Cited by 137 publications

References 38 publications

Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions

Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions

Relationship between surgeon volume and outcomes: a systematic review of systematic reviews

State of reporting of primary biomedical research: a scoping review protocol

Contact Info

Product

Resources

About