Dealing with Missing Predictor Values When Applying Clinical Prediction Models

Janssen, Kristel J.M.; Vergouwe, Yvonne; Donders, A. Rogier T.; Harrell, Frank E.; Chen, Qingxia; Grobbee, Diederick E.; Moons, Karel G.M.

doi:10.1373/clinchem.2008.115345

Cited by 130 publications

(122 citation statements)

References 27 publications

Supporting

Mentioning

107

Contrasting

Unclassified

Order By: Relevance

“…To minimize the effect of bias associated with selectively ignoring these patients, we imputed all missing data using multiple imputation techniques before undertaking analyses. 12,13 The diagnostic performance of gestalt and the Wells rule were compared using various methods. First, we quantified and compared the c statisticthe area under the curve (AUC)-of the receiver operating characteristic curve for gestalt and for the Wells rule.…”

Section: Statistical Analysesmentioning

confidence: 99%

Ruling Out Pulmonary Embolism in Primary Care: Comparison of the Diagnostic Performance of "Gestalt" and the Wells Rule

Hendriksen

Lucassen

Erkens

et al. 2016

The Annals of Family Medicine

View full text Add to dashboard Cite

PURPOSE Diagnostic prediction models such as the Wells rule can be used for safely ruling out pulmonary embolism (PE) when it is suspected. A physician's own probability estimate ("gestalt"), however, is commonly used instead. We evaluated the diagnostic performance of both approaches in primary care. METHODSFamily physicians estimated the probability of PE on a scale of 0% to 100% (gestalt) and calculated the Wells rule score in 598 patients with suspected PE who were thereafter referred to secondary care for definitive testing. We compared the discriminative ability (c statistic) of both approaches. Next, we stratified patients into PE risk categories. For gestalt, a probability of less than 20% plus a negative point-of-care d-dimer test indicated low risk; for the Wells rule, we used a score of 4 or lower plus a negative d-dimer test. We compared sensitivity, specificity, efficiency (percentage of low-risk patients in total cohort), and failure rate (percentage of patients having PE within the low-risk category).RESULTS With 3 months of follow-up, 73 patients (12%) were confirmed to have venous thromboembolism (a surrogate for PE at baseline). The c statistic was 0.77 (95% CI, 0.70-0.83) for gestalt and 0.80 (95% CI, 0.75-0.86) for the Wells rule. Gestalt missed 2 out of 152 low-risk patients (failure rate = 1.3%; 95% CI, 0.2%-4.7%) with an efficiency of 25% (95% CI, 22%-29%); the Wells rule missed 4 out of 272 low-risk patients (failure rate = 1.5%; 95% CI, 0.4%-3.7%) with an efficiency of 45% (95% CI, 41%-50%).CONCLUSIONS Combined with d-dimer testing, both gestalt using a cutoff of less than 20% and the Wells rule using a score of 4 or lower are safe for ruling out PE in primary care. The Wells rule is more efficient, however, and PE can be ruled out in a larger proportion of suspected cases.

show abstract

Section: Statistical Analysesmentioning

confidence: 99%

Ruling Out Pulmonary Embolism in Primary Care: Comparison of the Diagnostic Performance of "Gestalt" and the Wells Rule

Hendriksen

Lucassen

Erkens

et al. 2016

The Annals of Family Medicine

View full text Add to dashboard Cite

show abstract

“…30,31 As in model development, the use of routine care data was associated with missing predictor values; a larger fraction of patients had missing values in hospitals B and C. Multiple imputation was a feasible method of handling these missing observations in this context, and future work will need to identify the optimal method of applying the model in practice. 32 Interestingly, as a result of daily CSF sampling in hospital A, 83.3% of patients had no missing predictor information but model efficiency was lower compared to the other hospitals. This may in part be explained by the higher number of true infections and the increased likelihood of outlying measurements, especially because a relatively large fraction of patients incorrectly flagged as high risk were due to (incidental) anomalities in laboratory values.…”

Section: Discussionmentioning

confidence: 92%

Validation of an Automated Surveillance Approach for Drain-Related Meningitis: A Multicenter Study

Mourik

Troelstra

Sprenkel

et al. 2015

Infect. Control Hosp. Epidemiol.

View full text Add to dashboard Cite

objective. Manual surveillance of healthcare-associated infections is cumbersome and vulnerable to subjective interpretation. Automated systems are under development to improve efficiency and reliability of surveillance, for example by selecting high-risk patients requiring manual chart review. In this study, we aimed to validate a previously developed multivariable prediction modeling approach for detecting drain-related meningitis (DRM) in neurosurgical patients and to assess its merits compared to conventional methods of automated surveillance.methods. Prospective cohort study in 3 hospitals assessing the accuracy and efficiency of 2 automated surveillance methods for detecting DRM, the multivariable prediction model and a classification algorithm, using manual chart review as the reference standard. All 3 methods of surveillance were performed independently. Patients receiving cerebrospinal fluid drains were included (2012)(2013), except children, and patients deceased within 24 hours or with pre-existing meningitis. Data required by automated surveillance methods were extracted from routine care clinical data warehouses.results. In total, DRM occurred in 37 of 366 external cerebrospinal fluid drainage episodes (12.3/1000 drain days at risk). The multivariable prediction model had good discriminatory power (area under the ROC curve 0.91-1.00 by hospital), had adequate overall calibration, and could identify high-risk patients requiring manual confirmation with 97.3% sensitivity and 52.2% positive predictive value, decreasing the workload for manual surveillance by 81%. The multivariable approach was more efficient than classification algorithms in 2 of 3 hospitals.conclusions. Automated surveillance of DRM using a multivariable prediction model in multiple hospitals considerably reduced the burden for manual chart review at near-perfect sensitivity.Infect Control Hosp Epidemiol 2015;36(1): [65][66][67][68][69][70][71][72][73][74][75] introduction Electronically collected routine care data are increasingly employed to meet growing demands for reliable and timely information on healthcare-associated infection (HAI) rates. [1][2][3] For several decades, surveillance of and feedback regarding HAI rates, for example within national networks, have been fundamental components of infection prevention programs. [4][5][6][7] Traditionally, surveillance is performed by infection preventionists who manually review patient charts for the occurrence of targeted HAIs. This approach, however, is known to be labor intensive, effort dependent, and vulnerable to subjective interpretation. [8][9][10][11] Expansion of surveillance volume requirements and public reporting of HAI rates has stimulated the use of (semi)automated systems that combine various sources of data captured in electronic health records (EHRs) to support or replace manual surveillance. 12,13 Most of these automated surveillance systems aim to classify patients by their likelihood of having developed the targeted HAI and thereby restrict manual chart review to...

show abstract

“…Finally, this study can be biased because of the complete case analyses instead of analyzing after multiple imputations for missing variables, which is the preferred method of dealing with missing data. 22 However, because the respiratory status at 36 weeks PMA was missing in only 1.6% of the cases, it is unlikely that this will alter the results.…”

Section: Discussionmentioning

confidence: 96%

Accuracy of the Diagnosis of Bronchopulmonary Dysplasia in a Referral-Based Health Care System

Rossem¹,

Loo²,

Laan

et al. 2015

The Journal of Pediatrics

View full text Add to dashboard Cite

Dealing with Missing Predictor Values When Applying Clinical Prediction Models

Cited by 130 publications

References 27 publications

Ruling Out Pulmonary Embolism in Primary Care: Comparison of the Diagnostic Performance of "Gestalt" and the Wells Rule

Ruling Out Pulmonary Embolism in Primary Care: Comparison of the Diagnostic Performance of "Gestalt" and the Wells Rule

Validation of an Automated Surveillance Approach for Drain-Related Meningitis: A Multicenter Study

Accuracy of the Diagnosis of Bronchopulmonary Dysplasia in a Referral-Based Health Care System

Contact Info

Product

Resources

About