Predicting life expectancy with a long short-term memory recurrent neural network using electronic medical records

Abstract

Background Life expectancy is one of the most important factors in end-of-life decision making. Good prognostication for example helps to determine the course of treatment and helps to anticipate the procurement of health care services and facilities, or more broadly: facilitates Advance Care Planning. Advance Care Planning improves the quality of the final phase of life by stimulating doctors to explore the preferences for end-of-life care with their patients, and people close to the patients. Physicians, however, tend to overestimate life expectancy, and miss the window of opportunity to initiate Advance Care Planning. This research tests the potential of using machine learning and natural language processing techniques for predicting life expectancy from electronic medical records. Methods We approached the task of predicting life expectancy as a supervised machine learning task. We trained and tested a long short-term memory recurrent neural network on the medical records of deceased patients. We developed the model with a ten-fold cross-validation procedure, and evaluated its performance on a held-out set of test data. We compared the performance of a model which does not use text features (baseline model) to the performance of a model which uses features extracted from the free texts of the medical records (keyword model), and to doctors’ performance on a similar task as described in scientific literature. Results Both doctors and the baseline model were correct in 20% of the cases, taking a margin of 33% around the actual life expectancy as the target. The keyword model, in comparison, attained an accuracy of 29% with its prognoses. While doctors overestimated life expectancy in 63% of the incorrect prognoses, which harms anticipation to appropriate end-of-life care, the keyword model overestimated life expectancy in only 31% of the incorrect prognoses. Conclusions Prognostication of life expectancy is difficult for humans. Our research shows that machine learning and natural language processing techniques offer a feasible and promising approach to predicting life expectancy. The research has potential for real-life applications, such as supporting timely recognition of the right moment to start Advance Care Planning.

Review of Big Data Analytics, Artificial Intelligence and Nature-Inspired Computing Models towards Accurate Detection of COVID-19 Pandemic Cases and Contact Tracing

“…The ensembling model that combines the predictive capabilities of these models is proposed because it helps to assign equal weights to combined models [66]. For instance, LSTM and RNN have been applied to predict the life expectancy from electronic medical records [67]. Additionally, an LSTM and RNN model was used to predict the epidemic of COVID-19 [68].…”

Data-Driven Lexical Normalization for Medical Social Media

“…We manually constructed a decision process, inspired by the work by Beeksma et al [42], for detecting spelling mistakes. The optimal relative corpus frequency threshold determined for spelling correction in our earlier experiments is adopted.…”

“…Since domain-specific resources are scarce, one potential approach is to use the contextual information present in the corpus itself. Based on work by Beeksma et al [42], we tried to use the Part-of-Speech (POS) tags of the error or the POS tags of its neighbors to constrain correction candidates. However, as can be seen in Figure 5, adding these constraints reduced correction accuracy, although not significantly.…”

Early Prediction of 30-Day ICU Re-admissions Using Natural Language Processing and Machine Learning

“…The research proved that the combination of machine learning and natural language processing offers feasible pathways to making reliable predictions over complicated cases. [5]…”

“…We first extracted all patients' admission data from the table named ADMISSIONS. Next, we excluded patients with the admission type of 5 "NEWBORN", as the newborn patients' information might be archived in other database. The part of patients also has a high missing rate of discharge summaries.…”