Methodological quality and redundancy of systematic reviews that compare endarterectomy versus stenting for carotid stenosis

Abstract

A review of systematic reviews (SRs) and a critical appraisal study was conducted at Escola Paulista de Medicina, Universidade Federal de São Paulo (Unifesp). The objectives of this review are (1) to identify all published SRs comparing the effectiveness and safety of carotid endarterectomy (CEA) versus carotid artery stenting (CAS) for carotid artery stenosis, (2) to assess their methodological quality and (3) to compare the primary studies contained in each SR. We included 17 SRs published between 2005 and 2017. None of the SRs fulfilled all items of AMSTAR-2. The overall confidence in the results was graded as critically low for 16 SRs (94%) and low for 1 SR (6%). Five items were judged inadequate in all SRs: reference to a published protocol, explanation to the selection of study design, comprehensive search of the literature, methods for statistical combination of findings and consideration of the risk of bias on the results of meta-analysis. In total, 15 randomised clinical trials (RCTs) were included at least in one SR. The number of included RCTs in each SR was inconsistent (4 to 15) and was not related to the year of publication of the SR. Our study found redundant and low methodological quality SR comparing CAS versus CEA for carotid stenosis.

The quality of systematic reviews/meta‐analyses published in the field of bariatrics: A cross‐sectional systematic survey using AMSTAR 2 and ROBIS

“…The SR/MA's quality seems to be low in many fields of medicine. Among studies published in medicine to September 2019, in which the AMSTAR 2 was used as an instrument to evaluate the quality of SR/MA, 23 of 31 papers produced results similar to our study . In the six studies, the methodological quality of most (around 60%) of SRs was critical low to low .…”

A framework to characterise the reproducibility of meta‐analysis results with its application to direct oral anticoagulants in the acute treatment of venous thromboembolism

“…The phenomenon of meta-analysis replication, often described as redundancy in the literature, has been already highlighted in various domains. 5,6,8,16 Several arguments have been invoked to justify redundancy, such as: the wish to update the previous meta-analysis to include data from new high-quality trials, reluctance to extrapolate the results obtained to all patients in the absence of subgroup analyses, the need to assess an endpoint not evaluated in other meta-analyses or a change in endpoint definition or its assessment time. However, the term 'redundant' implicitly signifies that replication is unnecessary from a scientific point of view.…”

“…It is often described in the literature as redundancy and has already been observed in many fields of medical research. [3][4][5][6][7][8][9][10][11][12][13][14][15][16] Although investigating the same outcomes and including the same studies, these meta-analyses may differ in terms of the population of interest or the statistical methodology used, and these differences may influence their results. This phenomenon may be regarded as a 'vibration of effects' (VoE) and describes the extent to which the results of analyses could change according to the characteristics of the population included or the methodological choices made.…”

AMSTAR 2 Appraisal of Systematic Reviews and Meta-analyses in the Field of Heart Failure from High-impact Journals

“…Sixteen domains evaluated each step of the conduct of SRs and MAs. AMSTAR 2 has been previously used in various elds, including psychiatry, surgery, pediatrics, endocrinology, rheumatology, and cardiovascular disease, and many publications have reported substantial numbers of SRs with low to critically low quality [28][29][30][31][32][33]. Some authors called it a " oor effect" because of the lack of discrimination capacity of the tool, raising questions about its high standard and practical value [34].…”