CovRNN—A recurrent neural network model for predicting outcomes of COVID-19 patients: model development and validation using EHR data

Abstract

Background: Predicting outcomes of COVID-19 patients at an early stage is critical for optimized clinical care and resource management, especially during a pandemic. Although multiple machine learning models have been proposed to address this issue, based on the need for extensive data pre-processing and feature engineering, these models have not been validated or implemented outside of the original study site. Methods: In this study, we propose CovRNN, recurrent neural network (RNN)-based models to predict COVID-19 patients' outcomes, using their available electronic health record (EHR) data on admission, without the need for specific feature selection or missing data imputation. CovRNN is designed to predict three outcomes: in-hospital mortality, need for mechanical ventilation, and long length of stay (LOS >7 days). Predictions are made for time-to-event risk scores (survival prediction) and all-time risk scores (binary prediction). Our models were trained and validated using heterogeneous and de-identified data of 247,960 COVID-19 patients from 87 healthcare systems, derived from the Cerner® Real-World Dataset (CRWD). External validation was performed using three test sets (approximately 53,000 patients). Further, the transferability of CovRNN was validated using 36,140 de-identified patients' data derived from the Optum® de-identified COVID-19 Electronic Health Record v.1015 dataset (2007-2020). Findings: CovRNN shows higher performance than do traditional models. It achieved an area under the receiving operating characteristic (AUROC) of 93% for mortality and mechanical ventilation predictions on the CRWD test set (vs. 91.5% and 90% for light gradient boost machine (LGBM) and logistic regression (LR), respectively) and 86.5% for prediction of LOS > 7 days (vs. 81.7% and 80% for LGBM and LR, respectively). For survival prediction, CovRNN achieved a C-index of 86% for mortality and 92.6% for mechanical ventilation. External validation confirmed AUROCs in similar ranges. Interpretation: Trained on a large heterogeneous real-world dataset, our CovRNN model showed high prediction accuracy, good calibration, and transferability through consistently good performance on multiple external datasets. Our results demonstrate the feasibility of a COVID-19 predictive model that delivers high accuracy without the need for complex feature engineering.

Unveiling the Enigma: Exploring Risk Factors and Mechanisms for Psychotic Symptoms in Alzheimer’s Disease through Electronic Medical Records with Deep Learning Models

“…EMR data, which typically represents a patient's history as a sequence of visits with multiple events per visit, is well-suited for such sequence models as RNNs [64,65]. Recent studies indicated that simple-gated RNN models, such as Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTMs), when finely tuned using Bayesian Optimization, often deliver competitive outcomes [33]. Due to the limited sample size, we did not use Transformerbased models, which require a large amount of data for training.…”

“…In our previous study, we built a deep-learning-based model, DeepBiomarker, through modification of an established deep-learning framework, Pytorch_EHR [33,34]. In DeepBiomarker, we used diagnosis, medication use, and lab tests as the input, implemented data augmentation technologies to improve the model performance, and also integrated a perturbation-based approach [35] for risk factor identification.…”

AI-aided dynamic prediction of bleeding and ischemic risk after coronary stenting and subsequent DAPT

“…With respect to the implementation codes, we conducted data pre-processing on a Python-based pipeline, which resulted in patients’ comprehensive clinical profiling in a unified time sequence. We utilized the packages in scikit-learn 1.0.1 for RF and LR, an anaconda-supported package for LGBM, and an EHR-tailored predictive pipeline pytorch_ehr (32, 33) for GRU and RETAIN.…”

DeepBiomarker2: Prediction of alcohol and substance use disorder risk in post-traumatic stress disorder patients using electronic medical records and multiple social determinants of health

“…We adopted the Pytorch_EHR framework established by Zhi Group where Deep learning models based multiple recurrent neural networks 21 were used to analyze and predict clinical outcomes 22 . In conjunction with their algorithm and our previous DeepBiomarker model, we further modi ed the framework as DeepBiomarker2 by (a) integrating individual lab tests, SDoH parameters and medications along with the diagnosis groups as the input, so that we can assess the important clinical and nonclinical factors associated with ASUD risk; and (b) re ning contribution analysis 23 module by re ning the relative contribution analysis for the identi cation of key factors (see below for more details).…”