Clinical Prediction Rules: Challenges, Barriers, and Promise

Wallace, Emma; Johansen, Michael E.

doi:10.1370/afm.2303

Cited by 9 publications

(7 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 25 , 26 Some participants reported a lack of perceived value of CPRs or preference for clinical judgment, particularly among more experienced participants, which has been described in previous studies. 15 , 22 , 27 The finding that CPRs for RTIs may often be used to provide evidence to justify prescribing decisions to patients, or to confirm their own recommendation rather than aid decision-making, is consistent with findings on CPRs for other conditions. 28 In contrast to a survey study published in 2014 that found 76% of surveyed GPs reported that they had never heard of the Centor score, 15 the present study found that most participants were now aware of the score.…”

Section: Discussionsupporting

confidence: 63%

Understanding GPs’ views and experiences of using clinical prediction rules in the management of respiratory infections: a qualitative study

et al. 2021

View full text Add to dashboard Cite

Background:Respiratory tract infections (RTIs) account for 60% of antibiotic prescribing in primary care. Several clinical prediction rules (CPRs) have been developed to help reduce unnecessary prescribing for RTIs, but there are a lack of studies exploring whether or how these CPRs are being used in UK general practice.Aim:To explore UK GPs’ views and experiences with regards to RTI CPRs and to identify barriers and facilitators to their use in practice.Design & setting:A qualitative analysis of interviews with in-hours GPs working in the South and South West of England.Method:Semi-structured qualitative telephone interviews were conducted, digitally recorded, transcribed verbatim and analysed using an inductive thematic approach. Patient and public involvement representatives contributed to study design and interpretation of findings.Results:Thirty-two GPs were interviewed. Some CPRs were more commonly used than others. Participants used CPRs to facilitate patient-clinician discussion, confirm and support their decision, and document the consultation. GPs also highlighted concerns including lack of time, inability of CPRs to incorporate patient complexity, a shift in focus from the patient during consultations, and limited use in remote consultation (during the COVID-19 pandemic).Conclusion:This study highlights the need for user-friendly CPRs that are readily integrated into computer systems, and easily embedded into routine practice to complement clinical decision making. Existing CPRs need to be validated for other populations where demographics and clinical characteristics may differ as well different settings including remote consultations and self-assessment.

show abstract

Section: Discussionsupporting

confidence: 63%

Understanding GPs’ views and experiences of using clinical prediction rules in the management of respiratory infections: a qualitative study

et al. 2021

View full text Add to dashboard Cite

show abstract

“…11 Numerous prediction models to identify patients at risk of (unplanned) hospitalisations have been developed in various populations. 5 11-16 Several obstacles to good model performance have been identified, 17 but promising methodological advances have neither been able to provide a breakthrough in parametric modelling, 18 19 nor machine learning. 20 External validation in particular has proved to be a major challenge with regard to predictive performance.…”

Section: Introductionmentioning

confidence: 99%

Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity

et al. 2021

View full text Add to dashboard Cite

ObjectiveTo explore factors that potentially impact external validation performance while developing and validating a prognostic model for hospital admissions (HAs) in complex older general practice patients.Study design and settingUsing individual participant data from four cluster-randomised trials conducted in the Netherlands and Germany, we used logistic regression to develop a prognostic model to predict all-cause HAs within a 6-month follow-up period. A stratified intercept was used to account for heterogeneity in baseline risk between the studies. The model was validated both internally and by using internal-external cross-validation (IECV).ResultsPrior HAs, physical components of the health-related quality of life comorbidity index, and medication-related variables were used in the final model. While achieving moderate discriminatory performance, internal bootstrap validation revealed a pronounced risk of overfitting. The results of the IECV, in which calibration was highly variable even after accounting for between-study heterogeneity, agreed with this finding. Heterogeneity was equally reflected in differing baseline risk, predictor effects and absolute risk predictions.ConclusionsPredictor effect heterogeneity and differing baseline risk can explain the limited external performance of HA prediction models. With such drivers known, model adjustments in external validation settings (eg, intercept recalibration, complete updating) can be applied more purposefully.Trial registration numberPROSPERO id: CRD42018088129.

show abstract

“…9 Further, the application of prediction methods to data routinely collected in the electronic health record (EHR) provides a promising means to overcome some of the major barriers to the use of risk models. 10,11 For example, in addition to requiring manual ascertainment of variables, the previously reported DPP-based model required waist circumference and waist-to-hip ratio measurements that are not difficult to ascertain in routine practice. Herein, we describe development of a clinical prediction model using a hybrid approach that makes use of routinely collected EHR data to predict the risk of diabetes onset and clinical trial data to estimate unbiased risk-based effects of preventive interventions.…”

Section: Introductionmentioning

confidence: 99%

An Electronic Health Record Compatible Model to Predict Personalized Treatment Effects from the Diabetes Prevention Program: A Cross-Evidence Synthesis Approach Using Clinical Trial and Real World Data

Kent

Nelson

Pittas

et al. 2021

Preprint

View full text Add to dashboard Cite

BackgroundAn intensive lifestyle modification program or metformin pharmacotherapy reduced the risk of developing diabetes in patients at high risk, but are not widely used in the 88 million American adults with prediabetes.ObjectiveDevelop an electronic health record (EHR)-based risk tool that provides point-of-care estimates of diabetes risk to support targeting interventions to patients most likely to benefit.DesignCross-design synthesis: risk prediction model developed and validated in large observational database, treatment effect estimates from risk-based reanalysis of clinical trial data.SettingOutpatient clinics in US.PatientsRisk model development cohort: 1.1 million patients with prediabetes from the OptumLabs Data Warehouse (OLDW); validation cohort: distinct sample of 1.1 million patients in OLDW. Randomized clinical trial cohort: 3081 people from the Diabetes Prevention Program (DPP) study.InterventionsRandomization in the DPP: 1) an intensive program of lifestyle modification; 2) standard lifestyle recommendations plus 850 mg metformin twice daily; or 3) standard lifestyle recommendations plus placebo twice daily.ResultsEleven variables reliably obtainable from the EHR were used to predict diabetes risk. This model validated well in the OLDW (c-statistic = 0.76; observed 3-year diabetes rate was 1.8% in lowest-risk quarter and 19.6% in highest-risk quarter). In the DPP, the hazard ratio for lifestyle modification was constant across all levels of risk (HR = 0.43, 95% CI 0.35 – 0.53); while the HR for metformin was highly risk-dependent (HR HR = 1.1 [95% CI: 0.61 - 2.0] in the lowest-risk quarter vs. HR=0.45 [95% CI: 0.35 0.59] in the highest risk quarter). Fifty-three percent of the benefits of population-wide dissemination of the DPP lifestyle modification, and 76% of the benefits of population-wide metformin therapy can be obtained targeting the highest risk quarter of patients.LimitationsDifferences in variable definitions and in missingness across observational and trial settings may introduce estimation error in risk-based treatment effects.ConclusionAn EHR-compatible risk model might support targeted diabetes prevention to more efficiently realize the benefits of the DPP interventions.

show abstract

Clinical Prediction Rules: Challenges, Barriers, and Promise

Cited by 9 publications

References 22 publications

Understanding GPs’ views and experiences of using clinical prediction rules in the management of respiratory infections: a qualitative study

Understanding GPs’ views and experiences of using clinical prediction rules in the management of respiratory infections: a qualitative study

Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity

An Electronic Health Record Compatible Model to Predict Personalized Treatment Effects from the Diabetes Prevention Program: A Cross-Evidence Synthesis Approach Using Clinical Trial and Real World Data

Contact Info

Product

Resources

About