UpToDate adherence to GRADE criteria for strong recommendations: an analytical survey

Abstract

IntroductionUpToDate is widely used by clinicians worldwide and includes more than 9400 recommendations that apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. GRADE guidance warns against strong recommendations when certainty of the evidence is low or very low (discordant recommendations) but has identified five paradigmatic situations in which discordant recommendations may be justified.ObjectivesOur objective was to document the strength of recommendations in UpToDate and assess the frequency and appropriateness of discordant recommendations.DesignAnalytical survey of all recommendations in UpToDate.MethodsWe identified all GRADE recommendations in UpToDate and examined their strength (strong or weak) and certainty of the evidence (high, moderate or low certainty). We identified all discordant recommendations as of January 2015, and pairs of reviewers independently classified them either into one of the five appropriate paradigms or into one of three categories inconsistent with GRADE guidance, based on the evidence presented in UpToDate.ResultsUpToDate included 9451 GRADE recommendations, of which 6501 (68.8%) were formulated as weak recommendations and 2950 (31.2%) as strong. Among the strong, 844 (28.6%) were based on high certainty in effect estimates, 1740 (59.0%) on moderate certainty and 366 (12.4%) on low certainty. Of the 349 discordant recommendations 204 (58.5%) were judged appropriately (consistent with one of the five paradigms); we classified 47 (13.5%) as good practice statements; 38 (10.9%) misclassified the evidence as low certainty when it was at least moderate and 60 (17.2%) warranted a weak rather than a strong recommendation.ConclusionThe proportion of discordant recommendations in UpToDate is small (3.7% of all recommendations) and the proportion that is truly problematic (strong recommendations that would best have been weak) is very small (0.6%). Clinicians should nevertheless be cautious and look for clear explanations—in UpToDate and elsewhere—when guidelines offer strong recommendations based on low certainty evidence.

A unifying framework for improving health care

“…This means that we need to focus on patients' values and preferences, and local settings, a process that requires shared decision making . For instance, of over 10 000 GRADEd recommendations in the UpToDate online evidence‐based resource, two‐thirds are weak recommendations that give the clinicians the message to carefully consider the evidence in the context of patients' circumstances …”

Diagnostic, preventive and therapeutic evidence in obstetrics for the implementation of patient blood management: a systematic review protocol

“…The SoF will summarise the quality of the evidence, the certainty about the balance of benefits versus harms, the similarity in patients’ values and preferences and the costs of an intervention compared with the available alternatives. 37 …”

Moving towards online rheumatology education in the era of COVID-19

“…This is a portable resource that can aid even during clinical rounds [30]. An evaluation of the evidence presented in UpToDate points to the fact that less than 1% of the recommendations can be inappropriate [31]. ClinicalKey contains various updated guidelines, videos, and images.…”