Conflict between Guideline Methodologic Quality and Recommendation Validity: A Potential Problem for Practitioners

Watine, Joseph; Friedberg, Bruno; Nagy, É.; Ónody, Rita; Oosterhuis, Wytze P.; Bunting, Peter S.; Charet, Jean-Christophe; Horvath, Andrea R.

doi:10.1373/clinchem.2005.056952

Cited by 112 publications

(95 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…6 The most proximate outcomes would be each recommendation's alignment with the best evidence available. 13,22 Existing evaluation methods address the structure and process of guideline development and assume-erroneouslythat favorable outcomes will result. 10,11,14 For example, having a well-defined scope and purpose should produce recommendations that providers find applicable to common clinical situations.…”

Section: Discussionmentioning

confidence: 99%

“…However, another study found that guidelines with better methodological quality were not better aligned with a systematic literature review. 22 Even if a guideline's recommendations are perfectly concordant with the best available evidence, providers might still be dissatisfied if the recommendations are hard to apply or leave important clinical problems unaddressed because of gaps in the literature. The opposite conjecture, that issues with development resources and processes can adversely affect the intermediate outcome of perceived validity, appears supported by our data.…”

Section: Discussionmentioning

confidence: 99%

“…Published evidence is less developed for musculoskeletal disorders than some conditions. Our panelists assessed perceived validity rather than comparing guideline recommendations against a synthesis of evidence, as others have done, 22 but the latter would not reflect how providers examine guidelines in actual practice where the guidelines themselves must persuade readers of their validity. Attitudes toward individual guidelines vary greatly, and these findings might not apply to other conditions.…”

Section: Discussionmentioning

confidence: 99%

“…11 Internationally, AGREE is the preferred tool for assessing guidelines and its reliability has been demonstrated. [11][12][13]21 However, its validity remains unproved, 12,22 and it does not assess guideline content or the underlying evidence. 12,13,23 A question within AGREE's rigor-of-development domain stipulates that independent, external reviews by both clinical and methodological experts should be performed, but it does not specify what such reviews should entail.…”

Section: Appraising Technical Qualitymentioning

confidence: 99%

See 3 more Smart Citations

Rigorous Development does not Ensure that Guidelines are Acceptable to a Panel of Knowledgeable Providers

et al. 2007

View full text Add to dashboard Cite

BACKGROUND: Rigorous guideline development methods are designed to produce recommendations that are relevant to common clinical situations and consistent with evidence and expert understanding, thereby promoting guidelines' acceptability to providers. No studies have examined whether this technical quality consistently leads to acceptability. OBJECTIVE:To examine the clinical acceptability of guidelines having excellent technical quality. DESIGN AND MEASUREMENTS:We selected guidelines covering several musculoskeletal disorders and meeting 5 basic technical quality criteria, then used the widely accepted AGREE Instrument to evaluate technical quality. Adapting an established modified Delphi method, we assembled a multidisciplinary panel of providers recommended by their specialty societies as leaders in the field. Panelists rated acceptability, including "perceived comprehensiveness" (perceived relevance to common clinical situations) and "perceived validity" (consistency with their understanding of existing evidence and opinions), for ten common condition/therapy pairs pertaining to Surgery, physical therapy, and chiropractic manipulation for lumbar spine, shoulder, and carpal tunnel disorders. RESULTS:Five guidelines met selection criteria. Their AGREE scores were generally high indicating excellent technical quality. However, panelists found 4 guidelines to be only moderately comprehensive and valid, and a fifth guideline to be invalid overall. Of the topics covered by each guideline, panelists rated 50% to 69% as "comprehensive" and 6% to 50% as "valid". CONCLUSION:Despite very rigorous development methods compared with guidelines assessed in prior studies, experts felt that these guidelines omitted common clinical situations and contained much content of uncertain validity. Guideline acceptability should be independently and formally evaluated before dissemination.KEY WORDS: practice guidelines; quality assurance; health care; evidence-based medicine; attitude of health personnel; evaluation.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Appraising Technical Qualitymentioning

confidence: 99%

See 2 more Smart Citations

Rigorous Development does not Ensure that Guidelines are Acceptable to a Panel of Knowledgeable Providers

et al. 2007

View full text Add to dashboard Cite

show abstract

“…Differences in rigour of development do not necessarily result in different recommendations [14]. Thus, CPGs that fulfil the best methodological quality standards according to AGREE do not necessarily provide the most valid recommendations and vice versa.…”

Section: Variance In Quality Of Diabetes Guidelinesmentioning

confidence: 99%

Evidence base in guideline generation in diabetes

Mühlhauser¹,

Meyer

2013

Diabetologia

View full text Add to dashboard Cite

During recent years much emphasis has been on the validity, reliability, reproducibility, clinical applicability, clarity, multidisciplinary process, scheduled review and documentation of clinical practice guidelines (CPGs). Still, CPGs show substantial variance in methodological quality. The present paper mainly focuses on two aspects that are particularly critical and contemporary from the perspective of evidence-based medicine: patient centredness and shared decision making, and conflict of interest. Sophisticated patient and consumer involvement at all stages of CPG development could be judged as being the gold standard. However, co-opting patients or consumer representatives and using other techniques of active patient involvement does not replace individual patient preferences in clinical decision-making processes. Current CPGs do not meet patient needs, since they do not provide concise, easy-to-read summaries of the benefits and risks of medicines together with more comprehensive scientific data as a prerequisite for informed or shared decision making. The vast majority of CPG panels have a financial conflict of interest (COI) and under-reporting is common. Not all organisations producing CPGs have set up COI policies, and existing policies vary widely. To solve the problem, CPG experts have recommended that methodologists without any important COI should lead the development process and have primary responsibility. There is a lot of room for other improvements through network transnational activities in the field of CPG development. Waste of time and resources should be avoided through sharing published and unpublished data identified, appraised and extracted for guideline development. The EASD could provide such a clearing house.

show abstract

Factors Associated With High-Quality Guidelines for the Pharmacologic Management of Chronic Diseases in Primary Care

et al. 2019

View full text Add to dashboard Cite

for the Chronic Diseases and Informed Decisions (CHRONIDE) Group IMPORTANCE As the rate of publication of new and sometimes conflicting medical research increases, clinicians rely heavily on clinical practice guidelines (CPGs) to inform practice. However, CPGs are of widely variable quality, and there are no existing objective measures to rate the quality of CPGs.OBJECTIVE To systematically assess 421 CPGs for the management of common noncommunicable diseases in primary care using the validated Appraisal of Guidelines for Research and Evaluation Instrument, version II (AGREE-II) tool and elucidate the factors associated with quality of CPGs.EVIDENCE REVIEW MEDLINE, Embase, the Cochrane Library, and 12 websites for CPGs were searched for CPGs for the management of common noncommunicable diseases in primary care published between January 1, 2011, and August 30, 2017. The assessment of the quality of CPGs was performed by 3 appraisers using the 6 domains of the AGREE-II instrument. A multiple logistic regression was performed to identify factors associated with quality of CPGs.FINDINGS Of the 421 CPGs reviewed, 23.5% (99) were classified as high quality. Among included guidelines, clarity of presentation (70%) and scope and purpose (61%) had the highest median AGREE-II scores. The domains with the lowest median scores were applicability (22%) and rigor of development (33%). Factors associated with high-quality CPGs included having more than 20 authors (odds ratio, 9.08; 95% CI, 3.35-24.62), development at governmental institutions (odds ratio, 10.38; 95% CI, 2.72-39.60), and reporting funding (odds ratio, 10.34; 95% CI,. Year of publication, region, guideline version, and scope were not associated with quality among included CPGs.CONCLUSIONS AND RELEVANCE Primary care professionals and policymakers should be aware that CPGs in primary care are of widely variable quality, with less than 25% of included CPGs rated as high quality. High-quality CPGs were associated with a higher number of authors, governmental institutions, and the report of funding. Region of origin was not associated with quality of CPGs, which suggests that the improvement of the quality of CPGs should be an international concern.

show abstract

Conflict between Guideline Methodologic Quality and Recommendation Validity: A Potential Problem for Practitioners

Cited by 112 publications

References 35 publications

Rigorous Development does not Ensure that Guidelines are Acceptable to a Panel of Knowledgeable Providers

Rigorous Development does not Ensure that Guidelines are Acceptable to a Panel of Knowledgeable Providers

Evidence base in guideline generation in diabetes

Factors Associated With High-Quality Guidelines for the Pharmacologic Management of Chronic Diseases in Primary Care

Contact Info

Product

Resources

About