External validation of EPIC’s Risk of Unplanned Readmission model, the LACE+ index and SQLape as predictors of unplanned hospital readmissions: A monocentric, retrospective, diagnostic cohort study in Switzerland

Abstract

Introduction Readmissions after an acute care hospitalization are relatively common, costly to the health care system, and are associated with significant burden for patients. As one way to reduce costs and simultaneously improve quality of care, hospital readmissions receive increasing interest from policy makers. It is only relatively recently that strategies were developed with the specific aim of reducing unplanned readmissions using prediction models to identify patients at risk. EPIC’s Risk of Unplanned Readmission model promises superior performance. However, it has only been validated for the US setting. Therefore, the main objective of this study is to externally validate the EPIC’s Risk of Unplanned Readmission model and to compare it to the internationally, widely used LACE+ index, and the SQLAPE® tool, a Swiss national quality of care indicator. Methods A monocentric, retrospective, diagnostic cohort study was conducted. The study included inpatients, who were discharged between the 1st of January 2018 and the 31st of December 2019 from the Lucerne Cantonal Hospital, a tertiary-care provider in Central Switzerland. The study endpoint was an unplanned 30-day readmission. Models were replicated using the original intercept and beta coefficients as reported. Otherwise, score generator provided by the developers were used. For external validation, discrimination of the scores under investigation were assessed by calculating the area under the receiver operating characteristics curves (AUC). Calibration was assessed with the Hosmer-Lemeshow X2 goodness-of-fit test This report adheres to the TRIPOD statement for reporting of prediction models. Results At least 23,116 records were included. For discrimination, the EPIC´s prediction model, the LACE+ index and the SQLape® had AUCs of 0.692 (95% CI 0.676–0.708), 0.703 (95% CI 0.687–0.719) and 0.705 (95% CI 0.690–0.720). The Hosmer-Lemeshow X2 tests had values of p<0.001. Conclusion In summary, the EPIC´s model showed less favorable performance than its comparators. It may be assumed with caution that the EPIC´s model complexity has hampered its wide generalizability—model updating is warranted.

Electronic health record-based readmission risk model performance for patients undergoing outpatient parenteral antibiotic therapy (OPAT)

“…While the present study included one score per day, the Epic Readmission Risk Model V.1 calculates a readmission risk score every 4 hours for the hospitals within DUHS. The model variables include patient age, clinical diagnoses, laboratory values, medication numbers and classes, order types, and other healthcare use variables [ 9 , 15 ]. The resulting scores are provided as a continuous variable (from 0 to 100).…”

“…Others have reported 7, 14, and 30 days and that the utility of the model declined after 7 days post-discharge [ 14 ]. Unplanned readmissions within 30 days have also been utilized [ 15 ]. In addition to the timing, such endpoints capture both planned readmissions, as well as admissions unrelated to the target intervention.…”

The Independent Association of Plasma and Red Blood Cell Zinc Concentrations with Long-Term Outcomes of Hospitalized Patients

“…The LACE index is moderately discriminating ( c -statistic, 0.68) and is well calibrated [ 21 ]. It has been externally validated several times [ [21] , [22] , [23] , [24] ]. Risk of death or unplanned readmission increases as the LACE score increases.…”

The use of machine learning and artificial intelligence within pediatric critical care

“…Attempts at external validation have raised concerns about the validity of these models. 79,80 A different approach to CDSS is the augmentation of data visualization to assist physicians in better understanding trends in real time. One system that utilizes this approach is the Etiometry (Etiometry Inc., Boston, MA) risk analytics algorithm 81 that includes a data aggregation and visualization system in addition to a risk analytics engine.…”