Background and Objective: There is considerable actual and potential waste in research. Using evidence-based research (EBR) can ensure the value of a new study. The aim of this article, the third in a series, is to describe an EBR approach to putting research results into context.Study Design and Setting: EBR is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. In this third and final article of a series, we describe how to use the context of existing evidence to reach and present a trustworthy and useful conclusion when reporting results from a new clinical study.Results: We describe a method, the EBR approach, that by using a systematic and transparent consideration of earlier similar studies when interpreting and presenting results from a new original study will ensure usefulness of the conclusion.Conclusion: Using an EBR approach will improve the usefulness of a clinical study by providing the context to draw more valid conclusions and explicit information about new research needs.

BackgroundMany new clinical prediction rules are derived and validated. But the design and reporting quality of clinical prediction research has been less than optimal. We aimed to assess whether design characteristics of validation studies were associated with the overestimation of clinical prediction rules’ performance. We also aimed to evaluate whether validation studies clearly reported important methodological characteristics.MethodsElectronic databases were searched for systematic reviews of clinical prediction rule studies published between 2006 and 2010. Data were extracted from the eligible validation studies included in the systematic reviews. A meta-analytic meta-epidemiological approach was used to assess the influence of design characteristics on predictive performance. From each validation study, it was assessed whether 7 design and 7 reporting characteristics were properly described.ResultsA total of 287 validation studies of clinical prediction rule were collected from 15 systematic reviews (31 meta-analyses). Validation studies using case-control design produced a summary diagnostic odds ratio (DOR) 2.2 times (95% CI: 1.2–4.3) larger than validation studies using cohort design and unclear design. When differential verification was used, the summary DOR was overestimated by twofold (95% CI: 1.2 -3.1) compared to complete, partial and unclear verification. The summary RDOR of validation studies with inadequate sample size was 1.9 (95% CI: 1.2 -3.1) compared to studies with adequate sample size. Study site, reliability, and clinical prediction rule was adequately described in 10.1%, 9.4%, and 7.0% of validation studies respectively.ConclusionValidation studies with design shortcomings may overestimate the performance of clinical prediction rules. The quality of reporting among studies validating clinical prediction rules needs to be improved.

BackgroundResearchers should examine existing evidence to determine the need for a new study. It is unknown whether developers evaluate existing evidence to justify new cardiovascular clinical prediction rules (CPRs).ObjectiveWe aimed to assess whether authors of cardiovascular CPRs cited existing CPRs, why some authors did not cite existing CPRs, and why they thought existing CPRs were insufficient.MethodDerivation studies of cardiovascular CPRs from the International Register of Clinical Prediction Rules for Primary Care were evaluated. We reviewed the introduction sections to determine whether existing CPRs were cited. Using thematic content analysis, the stated reasons for determining existing cardiovascular CPRs insufficient were explored. Study authors were surveyed via e-mail and post. We asked whether they were aware of any existing cardiovascular CPRs at the time of derivation, how they searched for existing CPRs, and whether they thought it was important to cite existing CPRs.ResultsOf 85 derivation studies included, 48 (56.5%) cited existing CPRs, 33 (38.8%) did not cite any CPR, and four (4.7%) declared there was none to cite. Content analysis identified five categories of existing CPRs insufficiency related to: (1) derivation (5 studies; 11.4% of 44), (2) construct (31 studies; 70.5%), (3) performance (10 studies; 22.7%), (4) transferability (13 studies; 29.5%), and (5) evidence (8 studies; 18.2%). Authors of 54 derivation studies (71.1% of 76 authors contacted) responded to the survey. Twenty-five authors (46.3%) reported they were aware of existing CPR at the time of derivation. Twenty-nine authors (53.7%) declared they conducted a systematic search to identify existing CPRs. Most authors (90.7%) indicated citing existing CPRs was important.ConclusionCardiovascular CPRs are often developed without citing existing CPRs although most authors agree it is important. Common justifications for new CPRs concerned construct, including choice of predictor variables or relevance of outcomes. Developers should clearly justify why new CPRs are needed with reference to existing CPRs to avoid unnecessary duplication.

Background: Clinical prediction rules (CPRs) should be externally validated by independent researchers. Although there are many cardiovascular CPRs, most have not been externally validated. It is not known why some CPRs are externally validated by independent researchers and others are not. Methods: We analyzed cardiovascular risk CPRs included in a systematic review. Independent external validations were identified by forward citation searches of derivation studies. Time between the publication of a cardiovascular CPR and the first independent external validation was calculated. We assessed Kaplan-Meier estimates of the probability to have an independent external validation. Using univariable Cox regression, we explored whether characteristics of derivation (design, location, sample size, number of predictors, presentation format, validation in derivation), reporting (participants, predictors, outcomes, performance measure, information for risk calculation), and publication (journal impact factor) are associated with time to the first independent external validation. Results: Of 125 cardiovascular risk CPRs, 29 had an independent external validation. The median follow-up was 118 months (95% CI, 99-130). The 25th percentile of event time was 122 months (95% CI, 91-299). Cardiovascular risk CPRs from the USA were 4.15 times (95% CI, 1.89-9.13) more likely to have an independent external validation. Increasing the sample size of derivation by ten times was associated with a 2.32-fold (95% CI, 1.37-3.91) increase in the probability of having an independent external validation. CPRs presented with an internal validation tend to get an independent external validation sooner (HR = 1.73, 95% CI, 0.77-3.93). CPRs reporting all the information necessary for calculating individual risk were 2.65 (95% CI, 1.01-6.96) times more likely to have an independent external validation. Publishing a cardiovascular risk CPR in a journal that has one unit higher impact factor was associated with a 6% (95% CI, 3-9) higher likelihood of an independent external validation. Conclusions: The probability for cardiovascular risk CPRs to get an independent external validation was low even many years after their derivations. Authors of new cardiovascular risk CPRs should consider using adequate sample size, conducting an internal validation, and reporting all the information needed for risk calculation as these features were associated with an independent external validation.

BackgroundClinical prediction rules (CPRs) can help general practitioners (GPs) address challenges in cardiovascular disease. A survey published in 2014 evaluated GPs’ awareness and use of CPRs in the UK. However, many new CPRs have been published since and it is unknown which cardiovascular CPRs are currently recognised and used.AimTo identify cardiovascular CPRs recognised and used by GPs, and to assess how GPs’ familiarity and use have changed over time.Design & settingAn online survey of GPs in the UK was undertaken.MethodUsing comparable methods to the 2014 survey, GPs were recruited from a network of doctors in the UK. They were asked how familiar they were with cardiovascular CPRs, how frequently they used them, and why they used them. The results were compared with the 2014 survey.ResultsMost of 401 GPs were familiar with QRISK scores, ABCD scores, CHADS scores, HAS-BLED score, Wells scores for deep vein thrombosis, and Wells scores for pulmonary embolism. The proportions of GPs using these CPRs were 96.3%, 65.1%, 97.3%, 93.0%, 92.5%, and 82.0%, respectively. GPs’ use increased by 31.2% for QRISK scores, by 13.5% for ABCD scores, by 54.6% for CHADS scores, by 33.2% for Wells scores for deep vein thrombosis, and by 43.6% for Wells scores for pulmonary embolism; and decreased by 45.9% for the Joint British Societies (JBS) risk calculator, by 38.7% for Framingham risk scores, and by 8.7% for New Zealand tables. GPs most commonly used cardiovascular CPRs to guide therapy and referral.ConclusionThe study found GPs’ familiarity and use of cardiovascular CPRs changed substantially. Integrating CPRs into guidelines and practice software might increase familiarity and use.