The conduct and reporting of qualitative evidence syntheses in health and social care guidelines: a content analysis

Abstract

Background: This paper is part of a broader investigation into the ways in which health and social care guideline producers are using qualitative evidence syntheses (QESs) alongside more established methods of guideline development such as systematic reviews and meta-analyses of quantitative data. This study is a content analysis of QESs produced over a 5-year period by a leading provider of guidelines for the National Health Service in the UK (the National Institute for Health and Care Excellence) to explore how closely they match a reporting framework for QES. Methods: Guidelines published or updated between Jan 2015 and Dec 2019 were identified via searches of the National Institute for Health and Care excellence (NICE) website. These guidelines were searched to identify any QES conducted during the development of the guideline. Data relating to the compliance of these syntheses against a reporting framework for QES (ENTREQ) were extracted and compiled, and descriptive statistics used to provide an analysis of the of QES conduct, reporting and use by this major international guideline producer. Results: QES contributed, in part, to 54 out of a total of 192 guidelines over the five-year period. Although methods for producing and reporting QES have changed substantially over the past decade, this study found that there has been little change in the number or quality of NICE QESs over time. The largest predictor of quality was the centre or team which undertook the synthesis. Analysis indicated that elements of review methods which were similar to those used in quantitative systematic reviews tended to be carried out well and mostly matched the criteria in the reporting framework, but review methods which were more specific to a QES tended to be carried out less well, with fewer examples of criteria in the reporting framework being achieved. Conclusion: The study suggests that use, conduct and reporting of optimal QES methods requires development, as over time the quality of reporting of QES both overall, and by specific centres, has not improved in spite of clearer reporting frameworks and important methodological developments. Further staff training in QES methods may be helpful for reviewers who are more familiar with conventional forms of systematic review if the highest standards of QES are to be achieved. There seems potential for greater use of evidence from qualitative research during guideline development.

The move towards living systematic reviews and living guidelines in healthcare: consideration of the possibilities and challenges for living qualitative evidence syntheses

“…Since 2004, there has been a Cochrane Methods Group tasked with advising the Cochrane Collaboration on policy related to the synthesis of qualitative evidence and the integration of qualitative evidence with Cochrane effectiveness reviews. More recently, QES has become a part of the process of developing evidence-based health guidelines by organizations such as the UK National Institute for Health and Care Excellence (NICE) and by the World Health Organization (WHO) where they have been used by guideline panels to support their decision making [2][3][4]. This incorporation of QES into health guidelines has been made easier both by methodological developments in the ways that QES are undertaken (for example the introduction of GRADE CERQual [5]), and a drive by guidelineproducing organisations to consider the effects of patient preference, feasibility and acceptability on the broader effectiveness of a treatment or intervention when making guideline recommendations [3].…”