Best-worst scaling identified adequate statistical methods and literature search as the most important items of AMSTAR2 (A measurement tool to assess systematic reviews)

Leclercq, Victoria; Hiligsmann, Mickaël; Parisi, Gianni; Beaudart, Charlotte; Tirelli, Ezio; Bruyère, Olivier

doi:10.1016/j.jclinepi.2020.08.011

Cited by 17 publications

(13 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interestingly, our choice of items for potential revisions is closely aligned with importance ratings of AMSTAR2 items according to 242 experts in SR/meta‐analysis 27 . The experts rated the items on the appropriateness of statistical analyses, adequacy of the literature search and assessment of the risk of bias and heterogeneity as the most important items on AMSTAR2, 27 while we selected these items as candidates for possible revisions. Inspect individual AMSTAR2 items. Some overviews in our study did not derive the overall confidence ratings or, rather than focusing entirely on the poor confidence ratings, addressed the SR weaknesses based on the individual AMSTAR2 items.…”

Section: Discussionmentioning

confidence: 99%

“…Interestingly, our choice of items for potential revisions is closely aligned with importance ratings of AMSTAR2 items according to 242 experts in SR/meta-analysis. 27 The experts rated the items on the appropriateness of statistical analyses, adequacy of the literature search and assessment of the risk of bias and heterogeneity as the most important items on AMSTAR2, 27 while we selected these items as candidates for possible revisions. 3.…”

Section: Suggestions For Using Amstar2mentioning

confidence: 99%

See 1 more Smart Citation

The application of AMSTAR2 in 32 overviews of systematic reviews of interventions for mental and behavioural disorders: A cross‐sectional study

Santis

Lorenz

Lakeberg

et al. 2021

Research Synthesis Methods

View full text Add to dashboard Cite

A measurement tool to assess systematic reviews, version 2' (AMSTAR2) is a 16item tool to critically appraise systematic reviews (SRs) of healthcare interventions.This study aimed to assess the methods and outcomes of AMSTAR2 appraisals in overviews of SRs of interventions for mental and behavioural disorders. The crosssectional study was conducted using 32 overviews of SRs selected from three electronic databases in January 2021. Data items included overview and SR characteristics and AMSTAR2 appraisal methods and outcomes. Data were extracted by two authors independently and narratively synthesised using descriptive statistics (means ± SD and relative frequencies). SR characteristics were compared based on AMSTAR2 appraisal outcomes using chi-square tests. The 32 overviews appraised SRs of predominantly non-pharmacological interventions for mental disorders. AMSTAR2 appraisals were reported as confidence ratings in 25/32 overviews or individual item scores in 24/32 overviews. Most SRs/overview were non-Cochrane (mean = 94%), included RCTs only (mean = 77%) and were published before AMSTAR2 release (mean = 79%). The confidence ratings derived in 25 overviews for 349 SRs were predominantly critically low (68%). Confidence ratings were similar for SRs with RCTs only versus RCTs+non-RCTs or SRs published before versus after AMSTAR2 release, while Cochrane SRs received more high+moderate than low+critically low confidence ratings (p < 0.01). Confidence ratings derived based on AMSTAR2 do not differentiate among SRs of healthcare interventions except for Cochrane SRs that fulfil the criteria for high confidence ratings. AMSTAR2 items should be consulted to avoid common weaknesses in future SRs.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Suggestions For Using Amstar2mentioning

confidence: 99%

The application of AMSTAR2 in 32 overviews of systematic reviews of interventions for mental and behavioural disorders: A cross‐sectional study

Santis

Lorenz

Lakeberg

et al. 2021

Research Synthesis Methods

View full text Add to dashboard Cite

show abstract

“…3). Three systematic review features considered essential for acceptable systematic reviews – protocol, search, and RoB assessment of primary studies – were deficient in 93%, 69%, and 53% of our sample respectively 2,8 . Of note, the 28 systematic reviews with meta‐analysis fared no better than the rest of the systematic reviews; 100% were rated as critically low.…”

Section: Discussionmentioning

confidence: 90%

Quality appraisal of systematic reviews of interventions for children with cerebral palsy reveals critically low confidence

Kolaski

Logan

Goss

et al. 2021

Develop Med Child Neuro

View full text Add to dashboard Cite

Aim To evaluate the methodological quality of recent systematic reviews of interventions for children with cerebral palsy in order to determine the level of confidence in the reviews’ conclusions. Method A comprehensive search of 22 databases identified eligible systematic reviews with and without meta‐analysis published worldwide from 2015 to 2019. We independently extracted data and used A Measurement Tool to Assess Systematic Reviews‐2 (AMSTAR‐2) to appraise methodological quality. Results Eighty‐three systematic reviews met strict eligibility criteria. Most were from Europe and Latin America and reported on rehabilitative interventions. AMSTAR‐2 appraisal found critically low confidence in 88% (n=73) because of multiple and varied deficiencies. Only 7% (n=6) had no AMSTAR‐2 critical domain deficiency. The number of systematic reviews increased fivefold from 2015 to 2019; however, quality did not improve over time. Interpretation Most of these systematic reviews are considered unreliable according to AMSTAR‐2. Current recommendations for treating children with CP based on these flawed systematic reviews need re‐evaluation. Findings are comparable to reports from other areas of medicine, despite the general perception that systematic reviews are high‐level evidence. The required use of current widely accepted guidance for conducting and reporting systematic reviews by authors, peer reviewers, and editors is critical to ensure reliable, unbiased, and transparent systematic reviews. Confidence was critically low in the conclusions of 88% of systematic reviews about interventions for children with cerebral palsy (CP). Quality issues in the sample were not limited to systematic reviews of non‐randomized trials, or to those about certain populations of CP or interventions. The inclusion of meta‐analysis did not improve the level of confidence in these systematic reviews. Numbers of systematic reviews on this topic increased over the 5 search years but their methodological quality did not improve.

show abstract

“…However, none of the three items we found a difference for is critical according to the developers of AMSTAR 2. A recent survey involving a ranking exercise also found these three items not to be highranked when compared to the remaining AMSTAR 2 items (100). We also experienced di culties when assessing item 16 for industry-funded SRs.…”

Section: Discussionmentioning

confidence: 92%

Impact of industry sponsorship on the quality and interpretation of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016–2019

Pieper

Hellbrecht

Zhao

et al. 2021

Preprint

View full text Add to dashboard Cite

Background Systematic reviews (SRs) provide the highest level of evidence and inform evidence-based decision making in health care. Earlier studies found association with industry to be negatively associated with methodological quality of SRs. However, this has not been investigated in SRs on vaccines. Methods We performed a systematic literature search using MEDLINE and EMBASE in March 2020. The results were restricted to those published between 2016 to 2019 with no language restrictions. Study characteristics were extracted by one person and checked by an experienced reviewer. The methodological quality of the SRs was assessed with the AMSTAR 2 tool by multiple reviewers after a calibration exercise was performed. A summary score for each SR was calculated. The Mann-Whitney U-Test and Fisher’s Exact Test were performed to compare both groups. Results Out of 185 SRs that met all inclusion criteria, 27 SRs were industry funded. Those were matched with 30 non-industry funded SRs resulting in a total sample size of 57. The mean AMSTAR 2 summary score across all SRs was 0.49. Overall, the median AMSTAR 2 summary score was higher for the non-industry funded SRs than for the industry-funded SRs (0.62 vs. 0.36; p < .00001). Lower ratings for industry funded SRs were consistent across all but one AMSTAR 2 item, though significantly lower only for three specific items. Conclusion The methodological quality of SRs in vaccination is comparable to SRs in other fields, while it is still suboptimal. We are not able to provide a satisfactory explanation why industry funded SRs had a significantly lower methodological quality than non-industry funded SRs over recent years. Industry funding is an important indicator of methodological quality for vaccine SRs and should be carefully considered when appraising SR quality.

show abstract

Best-worst scaling identified adequate statistical methods and literature search as the most important items of AMSTAR2 (A measurement tool to assess systematic reviews)

Cited by 17 publications

References 21 publications

The application of AMSTAR2 in 32 overviews of systematic reviews of interventions for mental and behavioural disorders: A cross‐sectional study

The application of AMSTAR2 in 32 overviews of systematic reviews of interventions for mental and behavioural disorders: A cross‐sectional study

Quality appraisal of systematic reviews of interventions for children with cerebral palsy reveals critically low confidence

Impact of industry sponsorship on the quality and interpretation of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016–2019

Contact Info

Product

Resources

About