A Machine Learning Prediction Model of Respiratory Failure Within 48 Hours of Patient Admission for COVID-19: Model Development and Validation

Abstract

Background Predicting early respiratory failure due to COVID-19 can help triage patients to higher levels of care, allocate scarce resources, and reduce morbidity and mortality by appropriately monitoring and treating the patients at greatest risk for deterioration. Given the complexity of COVID-19, machine learning approaches may support clinical decision making for patients with this disease. Objective Our objective is to derive a machine learning model that predicts respiratory failure within 48 hours of admission based on data from the emergency department. Methods Data were collected from patients with COVID-19 who were admitted to Northwell Health acute care hospitals and were discharged, died, or spent a minimum of 48 hours in the hospital between March 1 and May 11, 2020. Of 11,525 patients, 933 (8.1%) were placed on invasive mechanical ventilation within 48 hours of admission. Variables used by the models included clinical and laboratory data commonly collected in the emergency department. We trained and validated three predictive models (two based on XGBoost and one that used logistic regression) using cross-hospital validation. We compared model performance among all three models as well as an established early warning score (Modified Early Warning Score) using receiver operating characteristic curves, precision-recall curves, and other metrics. Results The XGBoost model had the highest mean accuracy (0.919; area under the curve=0.77), outperforming the other two models as well as the Modified Early Warning Score. Important predictor variables included the type of oxygen delivery used in the emergency department, patient age, Emergency Severity Index level, respiratory rate, serum lactate, and demographic characteristics. Conclusions The XGBoost model had high predictive accuracy, outperforming other early warning scores. The clinical plausibility and predictive ability of XGBoost suggest that the model could be used to predict 48-hour respiratory failure in admitted patients with COVID-19.

Machine Learning Based Clinical Decision Support System for Early COVID-19 Mortality Prediction

“…The selected five features include neutrophils (%), hs-CRP, age, lymphocyte (%), and LDH. Each of these features have been identified as predictors of mortality associated with the COVID-19 disease (18, 22,24,25,[28][29][30][31][44][45][46]. Age has been identified as an important factor in COVID-19 disease progression and hence it has been included in all the models here (4-6, 22, 24, 25).…”

“…Each of these features have been identified as predictors of mortality associated with the COVID-19 disease ( 18 , 22 , 24 , 25 , 28 – 31 , 44 – 46 ). Age has been identified as an important factor in COVID-19 disease progression and hence it has been included in all the models here ( 4 – 6 , 22 , 24 , 25 ). Patients aged ≥60 years had a higher rate of respiratory failure and needed more prolonged treatment than those aged <60 years ( 4 ), implying that the elderly showed poorer response to treatments than the younger age group.…”

“…Different ML models have been proposed to predict risk of developing severe complications and mortality (15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27). This is important since there are limited resources compared to the increasing number of COVID-19 patients.…”

“…Common risk factors, included age, body mass index, and hypertension. Bolourani et al ( 22 ) developed ML models to predict respiratory failure within 48 h of patient admission for COVID-19. XGBoost model had the highest mean accuracy of 0.919 and AUC of 0.77.…”

Learning From Past Respiratory Infections to Predict COVID-19 Outcomes: Retrospective Study (Preprint)

“…We selected three decision tree-based algorithms because they have previously been applied to predict clinical events in patients with respiratory diseases based on EHR data [16,26,27]. We included models that were frequently applied for clinical prediction of severe patient outcomes [16,26,28]. In total, two steps were involved in model training: (1) using the training data set, a 10-fold cross-validation strategy was used to train the machine learning models, while grid search technique was used to search all combinations of hyperparameters and determine the best hyperparameters, and (2) using all training data, the models were retrained with the best hyperparameters (obtained in step 1).…”