Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study

Abstract

Background In the era of artificial intelligence, event prediction models are abundant. However, considering the limitation of the electronic medical record–based model, including the temporally skewed prediction and the record itself, these models could be delayed or could yield errors. Objective In this study, we aim to develop multiple event prediction models in intensive care units to overcome their temporal skewness and evaluate their robustness against delayed and erroneous input. Methods A total of 21,738 patients were included in the development cohort. Three events—death, sepsis, and acute kidney injury—were predicted. To overcome the temporal skewness, we developed three models for each event, which predicted the events in advance of three prespecified timepoints. Additionally, to evaluate the robustness against input error and delays, we added simulated errors and delayed input and calculated changes in the area under the receiver operating characteristic curve (AUROC) values. Results Most of the AUROC and area under the precision-recall curve values of each model were higher than those of the conventional scores, as well as other machine learning models previously used. In the error input experiment, except for our proposed model, an increase in the noise added to the model lowered the resulting AUROC value. However, the delayed input did not show the performance decreased in this experiment. Conclusions For a prediction model that was applicable in the real world, we considered not only performance but also temporal skewness, delayed input, and input error.

Rapid Response System Based on Graph Attention Network for Predicting In-Hospital Clinical Deterioration

“…The first one is an LSTMbased DEWS in [18] that consists of three RNN networks with LSTM cells. The second model is based on a Bi-LSTM architecture that is proposed in [22]. The last model that we mention in our experiment is a Bi-LSTM model with an attention mechanism (ABiLSTM) in [21].…”

“…RNN-based models have recently shown great potential for the medical and healthcare domain because of their capability to capture sequential patterns in time series [28]. The study in [22] developed prediction models for three events: sepsis, acute kidney injury (AKI), and death. Each model utilized bidirectional long short-term memory (Bi-LSTM) architecture to design binary classification models.…”

“…However, these deep learning (DL)-based models usually lack interpretability, as they are viewed as a black box that does not provide any insight about the features learned from the data [21]. There are also several approaches that utilize recurrent neural networks (RNNs) to capture sequence features in health records data [21][22]. These approaches require regularly spaced data points.…”

A Transfer Learning Approach to Correct the Temporal Performance Drift of Clinical Prediction Models: Retrospective Cohort Study

“…We designed TransferGBM based on several fundamental ideas. First, the base learner is GBM, which has been applied in a wide range of clinical prediction modeling studies [25,26]. GBM has been chosen because (1) it is robust to high-dimensional and collinearity data, (2) it can automatically process missing values, and (3) it embeds a unique feature selection scheme in the model training process, making its output more interpretable [20,27].…”

Self-supervised predictive coding and multimodal fusion advance patient deterioration prediction in fine-grained time resolution

“…Early prediction of vasopressor need can help clinicians efficiently prepare for an urgent vasopressor administration [2]. As a result, many studies have reported their early prediction systems using Electronic Health Records (EHR) [1,2,3].…”