Methodological quality of systematic reviews referenced in clinical practice guidelines for the treatment of opioid use disorder

Abstract

IntroductionWith efforts to combat opioid use disorder, there is an increased interest in clinical practice guidelines (CPGs) for opioid use disorder treatments. No literature exists examining the quality of systematic reviews used in opioid use disorder CPGs. This study aims to describe the methodological quality and reporting clarity of systematic reviews (SRs) used to create CPGs for opioid use disorder.MethodsFrom June to July 2016 guideline clearinghouses and medical literature databases were searched for relevant CPGs used in the treatment of opioid use disorder. Included CPGs must have been recognized by a national organization. SRs from the reference section of each CPG was scored by using AMSTAR (a measurement tool to assess the methodological quality of systematic reviews) tool and PRISMA (preferred reporting items for systematic reviews and meta-analyses) checklist.ResultsSeventeen CPGs from 2006–2016 were included in the review. From these, 57 unique SRs were extracted. SRS comprised 0.28% to 17.92% of all references found in the CPGs. All SRs obtained moderate or high methodological quality score on the AMSTAR tool. All reviews met at least 70% of PRISMA criteria. In PRISMA, underperforming areas included accurate title labeling, protocol registration, and risk of bias. Underperforming areas in AMSTAR included conflicts of interest, funding, and publication bias. A positive correlation was found between AMSTAR and PRISMA scores (r = .79).ConclusionAlthough the SRs in the CPGs were of good quality, there are still areas for improvement. Systematic reviewers should consult PRISMA and AMSTAR when conducting and reporting reviews. It is important for CPG developers to consider methodological quality as a factor when developing CPG recommendations, recognizing that the quality of systematic reviews underpinning guidelines does not necessarily correspond to the quality of the guideline itself.

Quality of systematic reviews in HIV: The case of clinical outcomes associated with patient medication adherence

“…We observed RoB underperformance across nearly all areas of SR methodology. One of the worst‐performing RoB items we observed, having an a priori protocol, was also observed by others 38 ; yet, it is an essential feature of the science 36 . The whole scientific method is dependent on having an a priori hypothesis and methods designed specifically to test that hypothesis.…”

“…Other domains with surprisingly low compliance were RoB assessment, for which numerous formal tools are available, 58,59 and interpretation of findings in the context of RoB assessment—the latter depending upon the former. Uncovering low‐quality original studies is an important role of SRs to prevent the “garbage in, garbage out phenomenon.” 53,60 Funding and conflicts of interest reporting also deserves more attention, based on our findings and that of others, 38 for both the review itself and the primary studies. Funding must be considered as there is considerable evidence of its impact on results 38,61 …”

“…We found inconsistent reporting and methodological quality in SRs on the clinical impact of poor ART adherence, despite the availability of numerous reporting guidelines (eg, PRISMA) and quality/RoB assessment tools (eg, AMSTAR 2, Critical Appraisal Skills Programme [CASP], Cochrane's RoB in Nonrandomized Studies of Interventions [ROBINS‐I], Newcastle Ottawa Scale, the Scottish Intercollegiate Guidelines Network [SIGN], and Mixed Methods Appraisal tool [MMAT]) 8,14,36 . Other researchers have reported similar findings in other medical specialties 8,37–47 . The poor quality we observed may explain the lack of reliance on these SRs in the US DHHS HIV treatment guideline 7 .…”

“…We observed search methods that were poorly planned and inadequately executed and/or reported. This finding is remarkable, given that the vast majority of potential threats to SR validity enter the process via failure to identify an unbiased sample of studies 38 . If search methods are not thorough and transparent, including a rigorous search for grey literature, users of the review can hardly trust the findings 54,55 .…”

The methodological quality of systematic reviews on the treatment of adult major depression needs improvement according to AMSTAR 2: A cross-sectional study

“…On the other hand, there are only few areas in medicine in which studies showed a good quality of SR. They include health literacy and cancer screening ( Sharma and Oremus, 2018 ) and SRs referenced in clinical practice guidelines for the treatment of opioid use disorder ( Ross et al., 2017 ).…”