Sepsis and severe sepsis contribute significantly to early treatment-related mortality after hematopoietic cell transplantation (HCT), with reported mortality rates of 30 and 55% due to severe sepsis, during engraftment admission, for autologous and allogeneic HCT, respectively. Since the clinical presentation and characteristics of sepsis immediately after HCT can be different from that seen in general population or those who are receiving non-HCT chemotherapy, detecting early signs of sepsis in HCT recipients becomes critical. Herein, we developed and validated a machine-learning based sepsis prediction model for patients who underwent HCT at City of Hope, using variables within the Electronic Health Record (EHR) data. We evaluated a consecutive case series of 1046 HCTs (autologous: n=491, allogeneic: n=555) at our center between 2014 and 2017. The median age at the time of HCT was 56 years (range: 18-78). For this analysis, the primary clinical event was sepsis diagnosis within 100 days post-HCT, identified based on - use of the institutional sepsis management order set and mention of "sepsis" in the progress notes. The time of sepsis order set was considered as time of sepsis for analyses. To train the model, 829 visits (104 septic and 725 non-septic) and their data were used, while 217 visits (31 septic and 186 non-septic) were used as a validation cohort. At each hour after HCT, when a new data point was available, 47 variables were calculated from each patient's data and a risk score was assigned to each time point. These variables consisted of patient demographics, transplant type, regimen intensity, disease status, Hematopoietic cell transplantation - specific comorbidity index, lab values, vital signs, medication orders, and comorbidities. For the 829 visits in the training dataset, the 47 variables were calculated at 220,889 different time points, resulting in a total of 10,381,783 data points. Lab values and vital signs were considered as changes from individual patient's baselines at each time point. The baseline for each lab value and vital sign were the last measured values before HCT. An ensemble of 20 random forest binary classification models were trained to identify and learn patterns of data for HCT patients at high risk for sepsis and differentiate them from patients at lower sepsis risk. To help the model learning patterns of data prior to sepsis, available data from septic patients' within 24 hours preceding diagnosis of sepsis was used. For 829 septic visits in the training data set, there were 5048 time points, each having 47 variables. Variable importance for the 20 models was assessed using Gini mean decrease accuracy method. The sum of importance values from each model was calculated for each variable as the final importance value. Figure 1a shows the importance of variables using this method. Testing the model on the validation cohort results in an AUC of 0.85 on the test dataset (Figure 1b). At a threshold of 0.6, our model was 0.32 sensitive and 0.96 specific. At this threshold, this model identified 10 out of 31 patients with a median lead time of 119.5 hours, of which 2 patients were flagged as high risk at the time of transplant and developed sepsis at 17 and 60 days post-HCT. The lead time is what truly sets this predictive model apart from detective models with organ failure or dysfunction or other deterioration metrics as their detection criteria. At a threshold of 0.4, our model has 0.9 sensitivity and 0.65 specificity. In summary, a machine-learning sepsis prediction model can be tailored towards HCT recipients to improve the quality of care, prevent sepsis associated-organ damage and decrease mortality post-HCT. Our model significantly outperforms widely used Modified Early Warning Score (MEWS), with AUC of 0.73 in general population. Possible application of our model include showing a "red flag" at a threshold of 0.6 (0.32 true positive rate and 0.04 false positive rate) for antibiotic initiation/modification, and a "yellow flag" at a threshold of 0.4 (0.9 true positive rate and 0.35 false positive rate) suggesting closer monitoring or less aggressive treatments for the patient. Figure 1. Figure 1. Disclosures Dadwal: MERK: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding, Speakers Bureau; Gilead: Research Funding; AiCuris: Research Funding; Shire: Research Funding.
Background: We recently reported that early CMV reactivation has adverse impact on blood counts recovery post CD34+ selected allogeneic hematopoietic stem cell transplantation (allo-HCT) (ASH 2017 poster 3319). Aim: To study the effects of CMV serostatus and CMV reactivation on blood counts recovery in the first-year post transplant among recipients of CD34 selected and unmodified graft. Methods: CMV viremia was assessed starting on day of engraftment or day 14 post-HCT (whichever occurred first) by quantitative PCR assay if recipient or donor were CMV positive. Per institutional guidelines, preemptive treatment with foscarnet, ganciclovir or valganciclovir is recommended when PCR is detected (<137 copies IU/ml) ×2 or more >137 copies IU/ml in CD34+ selected graft and PCR >500 copies IU/ml or rising in unmodified graft. Patients' blood counts data was included from the day of transplant and up to 1 year posttransplant. We studied the effects CMV serostatus and CMV reactivation by day 82 (last observed reactivation date) post allo-HCT on counts recovery in the first year post allo-HCT. Due to skewness, WBC, ANC, PLT and absolute reticulocyte count values were log transformed. Linear mixed effect models containing linear and cubic terms for time and random intercepts for each patient were fit, accounting for within patient correlation. Each model also included the day 82 CMV serostatus/reactivation status grouping variable. All p-values reflect an overall test for difference between the 3 serostatus groups. Results: The analysis included 317 patients who underwent allo-HCT between 4/2012-5/2016 for MDS, MPN, AML and ALL. CMV serostatus, reactivation rate and treatments are summarized in Figure 1. CMV serostatus was similar among recipients of CD34+ selected graft and unmodified graft, however, rate of reactivation and time to reactivation post-transplant were higher and earlier after CD34+ selected graft. In the CD34 selected group, patients who reactivated CMV had significantly lower ANC (P = .0001) and Hb (P = .023) and similar trend with Plt count, though that wasn't statistically significant (P = .07). CMV serostatus and reactivation had no effect on blood counts recovery in the unmodified group. Conclusion: CMV reactivation is more likely to occur in recipients of CD34+ selected grafts and also to occur earlier than in recipients of unmodified graft. In recipients of CD34+ selected graft it is associated with adverse impact on counts recovery. Active CMV infection or its treatment may contribute to the cytopenias observed. CMV prevention with a non-myelosuppressive antiviral has the potential of decreasing CMV infection and associated myelosuppression.
Sepsis contributes significantly to early treatment-related mortality after hematopoietic cell transplantation (HCT). Since the clinical presentation and characteristics of sepsis immediately after HCT can be different from that seen in general population or those who are receiving non-HCT chemotherapy, detecting early signs of sepsis in HCT recipients becomes critical. Herein, we extended our earlier analyses (Dadwal et al. ASH 2018) and evaluated a consecutive case series of 1806 patients who underwent HCT at City of Hope (2014-2017) to develop a machine-learning sepsis prediction model for HCT recipients, namely Early Sepsis Prediction/Identification for Transplant Recipients (ESPRIT) using variables within the Electronic Health Record (EHR) data. The primary clinical event was sepsis diagnosis within 100 days post-HCT, identified based on the use of the institutional "sepsis management order set" and mention of "sepsis" in the progress notes. The time of sepsis order set was considered as time of sepsis for the analyses. Data from 2014 to 2016 (108 visits with and 1315 visits without sepsis, 8% sepsis prevalence) were used as the training set and data from 2017 (24 visits with and 359 visits without sepsis, 6.6% sepsis prevalence) were kept as the holdout dataset for testing the model. From each patient visit, 61 variables were collected with a total of 862,009 lab values, 3,284,561 vital sign values and 249,982 medication orders for 1806 visits over the duration of HCT hospitalization (median: 24.1 days, range: 7-304). An ensemble of 100 random forest classification models were used to develop the prediction model. Last Observation Carried Forward (LOCF) imputation was done to attribute the missing values with the last observed value of that variable. For model development and optimization, we applied a 5-fold stratified cross validation on the training dataset. Variable importance for the 100 models was assessed using Gini mean decrease accuracy method value, which was averaged to produce the final variable importance. HCT was autologous in 798 and allogeneic in 1008 patients. Ablative conditioning regimen was delivered to 97.3% and 38.3% of patients in autologous and allogeneic groups, respectively. When the impact of "sepsis" was analyzed as a time-dependent variable, sepsis development was associated with increased mortality (HR=2.79, 95%CI: 2.14-3.64, p<0.001) by multivariable Cox regression model. Retrospective evaluation at 0, 4, 8 and 12 hours pre-sepsis showed area under the ROC curves (AUCs) of 0.98, 0.91, 0.90 and 0.85, respectively (Fig 1a), outperforming the widely used Modified Early Warning Score (MEWS) (Fig 1b). We then simulated our ESPRIT's performance in the unselected real-world data by running the model every hour from admit to sepsis or discharge, whichever occurred first. This process created an hourly risk score from admit to sepsis or discharge. ESPRIT achieved an AUC of 0.83 on the training and AUC of 0.82 on the holdout test dataset (Fig 2). An example of risk over time for a septic patient that was identified by the model with 27 hours lead time at threshold of 0.6 is shown in Fig 3. With at risk threshold of 0.6 (sensitivity: 0.4, specificity: 0.93), ESPRIT had a median lead time of 35 and 47 hours on training and holdout test data, respectively. This model allows users to select any threshold (with specific false positive/negative rate expected for a given population) to be used for specific purposes. For example, a red flag can be assigned to a patient when the risk passes the threshold of 0.6. At this threshold the false positive rate is only 7% and true positive rate is 40%. Then a yellow flag can be assigned at the threshold of 0.4, with which the model has higher (38%) false positive rate but also a high (90%) true positive rate. Using this two-step assessment/intervention system (red flag as an alarm and yellow flag as a warning sign to examine the patient to rule out sepsis), the model would achieve 90% sensitivity and 93% specificity in practice and overcome the low positive predictive value due to the rare incidence of sepsis. In summary, we developed and validated a novel machine learning monitoring system for sepsis prediction in HCT recipients. Our data strongly support further clinical validation of the ESPRIT model as a method to provide real-time sepsis predictions, and timely initiation of preemptive antibiotics therapy according to the predicted risks in the era of EHR. Disclosures Dadwal: Ansun biopharma: Research Funding; SHIRE: Research Funding; Janssen: Membership on an entity's Board of Directors or advisory committees; Merck: Membership on an entity's Board of Directors or advisory committees; Clinigen: Membership on an entity's Board of Directors or advisory committees. Nakamura:Kirin Kyowa: Other: support for an academic seminar in a university in Japan; Merck: Membership on an entity's Board of Directors or advisory committees; Celgene: Other: support for an academic seminar in a university in Japan; Alexion: Other: support to a lecture at a Japan Society of Transfusion/Cellular Therapy meeting .
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.