External Validation of Three Prediction Tools for Patients at Risk of a Complicated Course of Clostridium difficile Infection: Disappointing in an Outbreak Setting

Abstract

OBJECTIVE Estimating the risk of a complicated course of Clostridium difficile infection (CDI) might help doctors guide treatment. We aimed to validate 3 published prediction models: Hensgens (2014), Na (2015), and Welfare (2011). METHODS The validation cohort comprised 148 patients diagnosed with CDI between May 2013 and March 2014. During this period, 70 endemic cases of CDI occurred as well as 78 cases of CDI related to an outbreak of C. difficile ribotype 027. Model calibration and discrimination were assessed for the 3 prediction rules. RESULTS A complicated course (ie, death, colectomy, or ICU admission due to CDI) was observed in 31 patients (21%), and 23 patients (16%) died within 30 days of CDI diagnosis. The performance of all 3 prediction models was poor when applied to the total validation cohort with an estimated area under the curve (AUC) of 0.68 for the Hensgens model, 0.54 for the Na model, and 0.61 for the Welfare model. For those patients diagnosed with CDI due to non-outbreak strains, the prediction model developed by Hensgens performed the best, with an AUC of 0.78. CONCLUSION All 3 prediction models performed poorly when using our total cohort, which included CDI cases from an outbreak as well as endemic cases. The prediction model of Hensgens performed relatively well for patients diagnosed with CDI due to non-outbreak strains, and this model may be useful in endemic settings. Infect Control Hosp Epidemiol 2017;38:897-905.

Systemic Inflammatory Mediators Are Effective Biomarkers for Predicting Adverse Outcomes in Clostridioides difficile Infection

“…CDI is associated with an increased risk of mortality, and at present, we are inadequately determining who will experience adverse outcomes. Multiple models have been produced to address this problem, including those utilizing electronic medical records, standard laboratory tests, and medical history (6)(7)(8)(9)(10)(11)(12). However, these models have met with limited success in external validation, and there is room for improvement in predictive ability of CDI adverse outcomes.…”

“…An ideal model would identify cases of CDI at the time of diagnosis that are progressing toward severe systemic disease, so that treatments to halt disease progression can be started. Models built from baseline clinical variables or standard laboratory measurements have met with limited success in accurately predicting adverse outcomes, or they do not validate externally (6)(7)(8)(9)(10)(11)(12). Therefore, we set out to determine if predictive models built from a panel of multiple inflammatory mediators measured at diagnosis of CDI can accurately predict adverse outcomes, specifically, 30-day all-cause mortality and DRCs defined as ICU admission, colectomy, and/or death attributed to CDI.…”

External validation of clinical prediction rules for complications and mortality following Clostridioides difficile infection

“…However, a decrease in performance is not unusual in the external validation of CPRs [7]. For example, the scores of Na and Hensgens [25, 26] performed similarly (AUC = 0.54 and 0.68 respectively) in a second validation cohort of 148 patients during an outbreak of R027 strains [33]. The effect of strain on CDI outcome is still controversial in observational studies [34], and we did not find any significant association with cCDI in our cohort [11], despite the high frequency of R027.…”

“…Other studies have attempted external validation of indices for severe CDI course on much smaller sample sizes [33, 37–39]. While we identified relevant studies through a rigorous systematic review of the literature and included only studies with clear derivation methodology and at least one internal validation assessment.…”

“…While we identified relevant studies through a rigorous systematic review of the literature and included only studies with clear derivation methodology and at least one internal validation assessment. Only van Beurden et al [33] used a standardized selection of studies.…”