Assessing the Quality of Systematic Reviews in Healthcare Using AMSTAR and AMSTAR2

Santis, Karina Karolina De; Kaplan, Ilkay

doi:10.1027/2151-2604/a000397

Cited by 10 publications

(15 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As they get more complex and challenging, we learn more about the process [29] by applying multiple instruments to judge the quality of systematic reviews or meta-analyses, the most prominent among them being the AMSTAR-score [30], the ROBIS-tool [31], and the AMSTAR-2-instrument [32]. The latter two have been found to give similar results [33], whereas the AMSTAR-2-instrument outperformed its predecessor [34], which already identified 75% of all investigated systematic reviews and meta-analyses to be only of poor or fair quality [13]. The original AMSTARscore has eleven items [30] used to assess the quality of the systematic review by addressing several points that would have a relevant effect on the robustness of findings in the systematic review and meta-analysis.…”

Section: Discussionmentioning

confidence: 99%

Meta-analyses in paediatric surgery are often fragile: implications and consequences

2021

View full text Add to dashboard Cite

Purpose Meta-analyses occupy the highest level of evidence and thereby guide clinical decision-making. Recently, randomised-controlled trials were evaluated for the robustness of their findings by calculating the fragility index. The fragility index is the number of events that needs to be added to one treatment arm until the statistical significance collapses. We, therefore, aimed to evaluate the robustness of paediatric surgical meta-analyses. Methods We searched MEDLINE for paediatric surgical meta-analyses in the last decade. All meta-analyses on a paediatric surgical condition were eligible for analysis if they based their conclusion on a statistically significant meta-analysis. Results We screened 303 records and conducted a full-text evaluation of 60 manuscripts. Of them, 39 were included in our analysis that conducted 79 individual meta-analyses with significant results. Median fragility index was 5 (Q25–Q75% 2–11). Median fragility in relation to included patients was 0.77% (Q25–Q75% 0.29–1.87%). Conclusion Paediatric surgical meta-analyses are often fragile. In almost 60% of results, the statistical significance depends on less than 1% of the included population. However, as the fragility index is just a transformation of the P value, it basically conveys the same information in a different format. It therefore should be avoided.

show abstract

Section: Discussionmentioning

confidence: 99%

Meta-analyses in paediatric surgery are often fragile: implications and consequences

2021

View full text Add to dashboard Cite

show abstract

“…De Santis et al compared scores on AMSTAR and AMSTAR 2 with the following justification [quote]: “ Since AMSTAR and AMSTAR2 differ substantially, it remains unclear if they produce similar quality ratings for systematic reviews in healthcare ” [ 26 ]. Jeyaraman et al used both AMSTAR and AMSTAR 2 without any comments or explanations for using both tools [ 27 ].…”

Section: Resultsmentioning

confidence: 99%

Adopting AMSTAR 2 critical appraisal tool for systematic reviews: speed of the tool uptake and barriers for its adoption

Bojčić

Todorić

Puljak

2022

BMC Med Res Methodol

View full text Add to dashboard Cite

Background In 2007, AMSTAR (A MeaSurement Tool to Assess systematic Reviews), a critical appraisal tool for systematic reviews (SRs), was published, and it has since become one of the most widely used instruments for SR appraisal. In September 2017, AMSTAR 2 was published as an updated version of the tool. This mixed-methods study aimed to analyze the extent of the AMSTAR 2 uptake and explore potential barriers to its uptake. Methods We analyzed the frequency of AMSTAR or AMSTAR 2 use in articles published in 2018, 2019 and 2020. We surveyed authors who have used AMSTAR but not AMSTAR 2 in the analyzed time frame to identify their reasons and barriers. The inclusion criterion for those authors was that the month of manuscript submission was after September 2017, i.e. after AMSTAR 2 was published. Results We included 871 studies. The majority (N = 451; 52%) used AMSTAR 2, while 44% (N = 382) used AMSTAR, 4% (N = 31) used R-AMSTAR and others used a combination of tools. In 2018, 81% of the analyzed studies used AMSTAR, while 16% used AMSTAR 2. In 2019, 52% used AMSTAR, while 44% used AMSTAR 2. Among articles published in 2020, 28% used AMSTAR, while AMSTAR 2 was used by 69%. An author survey indicated that the authors did not use AMSTAR 2 mostly because they were not aware of it, their protocol was already established, or data collection completed at the time when the new tool was published. Barriers towards AMSTAR 2 use were lack of quantitative assessment, insufficient awareness, length, difficulties with a specific item. Conclusion In articles published in 2018-2020, that were submitted to a journal after AMSTAR 2 tool was published, almost half of the authors (44%) still used AMSTAR, the old version of the tool. However, the use of AMSTAR has been declining in each subsequent year. Our survey indicated that editors and peer-reviewers did not ask the authors to use the new version of the tool. Few barriers towards using AMSTAR 2 were identified, and thus it is anticipated that the use of the old version of AMSTAR will continue to decline.

show abstract

“…This step was necessary to prevent any research waste. According to meta-research [19][20][21], many reviews of interventions in the health context are redundant because they either address the same aims as those addressed by other existing reviews or cite the same primary studies. This problem is so extensive that some reviews do not include any unique primary studies that are not cited in other reviews, and there are as many reviews as or even more reviews than there are primary studies in some fields [19][20][21].…”

Section: Protocol and Registrationmentioning

confidence: 99%

“…According to meta-research [19][20][21], many reviews of interventions in the health context are redundant because they either address the same aims as those addressed by other existing reviews or cite the same primary studies. This problem is so extensive that some reviews do not include any unique primary studies that are not cited in other reviews, and there are as many reviews as or even more reviews than there are primary studies in some fields [19][20][21]. The procedure of checking if a new review is required prior to study registration may be especially necessary in rapidly developing fields, such as digitally supported interventions, or when addressing commonly investigated outcomes, such as PA promotion.…”

Section: Protocol and Registrationmentioning

confidence: 99%

Evaluation of Digital Interventions for Physical Activity Promotion: Protocol for a Scoping Review

Santis¹,

Jahnel²,

Mergenthal³

et al. 2022

JMIR Res Protoc

Self Cite

View full text Add to dashboard Cite

Background Digital interventions (DIs) could support physical activity (PA) promotion, according to recent reviews. However, it remains unclear if and how DIs for PA promotion are evaluated; thus, it is unclear if they support behavior change in real-world settings. A mapping of evidence from published reviews is required to focus on the evaluation of DIs for PA promotion. Objective The aim of our study is to investigate evaluation strategies for any outcome in the context of DIs for PA promotion by conducting a scoping review of published reviews. Methods Our scoping review adheres to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. The information sources include bibliographic databases (MEDLINE, PsycINFO, and CINAHL) and the bibliographies of the selected studies. The electronic search strategy was developed and conducted in collaboration with an experienced database specialist. The electronic search was conducted in English with no limits up to March 19, 2021, for sources with the terms digital intervention AND evaluation AND physical activity in titles or abstracts. After deduplication, 300 reviews selected from 4912 search results were assessed for eligibility by 2 authors working independently. The inclusion criteria were (1) healthy or clinical samples (population), (2) DIs for PA promotion (intervention), (3) comparisons to any other intervention or no intervention (comparison), (4) evaluation strategies (methods, results, or frameworks) for any outcome in the context of DIs for PA promotion (outcome), and (5) any published review (study type). According to the consensus reached during a discussion, 40 reviews met the inclusion criteria—36 from the electronic search and 4 from the manual search of the bibliographies of the 36 reviews. All reviews reported the evaluation strategies for any outcomes in the context of DIs for PA promotion in healthy or clinical samples. Data coding and the quality appraisal of systematic reviews are currently being performed independently by 2 authors. Results Our scoping review includes data from 40 published reviews (1 rapid review, 9 scoping reviews, and 30 systematic reviews). The focus of data coding is on evaluation strategies in the context of DIs for PA promotion and on the critical appraisal of the included systematic reviews. The final consensus regarding all data is expected in early 2022. Conclusions Interventions for PA promotion that are supported by digital technologies require evaluation to ensure their efficacy in real-world settings. Our scoping review is needed because it addresses novel objectives that focus on such evaluations and are not answered in the published reviews identified in our search. The evaluation strategies addressing DIs for PA promotion will be mapped to synthesize the results that have been reported in published reviews so far. International Registered Report Identifier (IRRID) DERR1-10.2196/35332

show abstract

Assessing the Quality of Systematic Reviews in Healthcare Using AMSTAR and AMSTAR2

Cited by 10 publications

References 17 publications

Meta-analyses in paediatric surgery are often fragile: implications and consequences

Meta-analyses in paediatric surgery are often fragile: implications and consequences

Adopting AMSTAR 2 critical appraisal tool for systematic reviews: speed of the tool uptake and barriers for its adoption

Evaluation of Digital Interventions for Physical Activity Promotion: Protocol for a Scoping Review

Contact Info

Product

Resources

About