Methodological quality of meta-analyses indexed in PsycINFO: leads for enhancements: a meta-epidemiological study

Abstract

ObjectivesMeta-analyses (MAs) are often used because they are lauded to provide robust evidence that synthesises information from multiple studies. However, the validity of MA conclusions relies on the procedural rigour applied by the authors. Therefore, this meta-research study aims to characterise the methodological quality and meta-analytic practices of MAs indexed in PsycINFO.DesignA meta-epidemiological study.ParticipantsWe evaluated a random sample of 206 MAs indexed in the PsycINFO database in 2016.Primary and secondary outcomesTwo authors independently extracted the methodological characteristics of all MAs and checked their quality according to the 16 items of the A MeaSurement Tool to Assess systematic Reviews (AMSTAR2) tool for MA critical appraisal. Moreover, we investigated the effect of mentioning Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) on the methodological quality of MAs.ResultsAccording to AMSTAR2 criteria, 95% of the 206 MAs were rated as critically low quality. Statistical methods were appropriate and publication bias was well evaluated in 87% and 70% of the MAs, respectively. However, much improvement is needed in data collection and analysis: only 11% of MAs published a research protocol, 44% had a comprehensive literature search strategy, 37% assessed and 29% interpreted the risk of bias in the individual included studies, and 11% presented a list of excluded studies. Interestingly, the explicit mentioning of PRISMA suggested a positive influence on the methodological quality of MAs.ConclusionThe methodological quality of MAs in our sample was critically low according to the AMSTAR2 criteria. Some efforts to tremendously improve the methodological quality of MAs could increase their robustness and reliability.

The application of AMSTAR2 in 32 overviews of systematic reviews of interventions for mental and behavioural disorders: A cross‐sectional study

“…16 It is disappointing that the overall confidence in most SRs of interventions for mental and behavioural disorders is low to critically low as already shown in various clinical fields. 4,5,8,[11][12][13][14][15] While authors of SRs are primarily responsible for the quality of their work, the 'gatekeepers' of scientific publishing (peer-reviewers and journal editors) are equally responsible for guiding SR authors through the publication process, especially if the authors are new in the field of research synthesis or are not English speakers.…”

“…SRs are typically appraised to identify common problems, including methodological limitations and/or poor reporting. 3,4,[13][14][15][20][21][22][23][24][25] While the focus of AMSTAR2 is on the methodological quality, this tool is highly sensitive to poor reporting. Specifically, inadequate reporting of only 2/16 items can be sufficient to assign a critically low confidence rating to a SR.…”

“…First, AMSTAR2 tends to assign the same (mostly low to critically low) ratings to SRs of various healthcare interventions in different clinical fields. 4,5,8,[11][12][13][14][15] Second, AMSTAR2 users derive the overall ratings using different methods that are often not explicitly reported. 16 It is possible that both findings are related in that AMSTAR2 users attempt to improve the discriminatory power of the tool by selecting specific critical items according to their research questions, as advised by AMSTAR2 developers.…”

Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention

“…Despite the fact that several previous studies reported insufficient assessment of RoB of primary studies included in SR/MAs [ 21 , 28 , 29 , 30 , 31 ], we did not identify any studies that would formally test such a variable as a predictor of SR quality. However, previous studies showed that involvement of a ‘research methodologist’ predicts a higher quality of SRs.…”

Systematic Reviews and Meta-analysis Published in Indexed Portuguese Medical Journals: Time Trends and Critical Appraisal.

“…Most studies report similar results, with a large number of SRs included being rated as low or critical low quality. (23)(24)(25)(26)(27) Currently, there are some quality appraisal tools and guidelines that might help authors and peer reviewers to improve and assess methodological report and review of SRs. The AMSTAR II is one of these tools.…”