Predicting the risk of emergency admission with machine learning: Development and validation using linked electronic health records

Abstract

BackgroundEmergency admissions are a major source of healthcare spending. We aimed to derive, validate, and compare conventional and machine learning models for prediction of the first emergency admission. Machine learning methods are capable of capturing complex interactions that are likely to be present when predicting less specific outcomes, such as this one.Methods and findingsWe used longitudinal data from linked electronic health records of 4.6 million patients aged 18–100 years from 389 practices across England between 1985 to 2015. The population was divided into a derivation cohort (80%, 3.75 million patients from 300 general practices) and a validation cohort (20%, 0.88 million patients from 89 general practices) from geographically distinct regions with different risk levels. We first replicated a previously reported Cox proportional hazards (CPH) model for prediction of the risk of the first emergency admission up to 24 months after baseline. This reference model was then compared with 2 machine learning models, random forest (RF) and gradient boosting classifier (GBC). The initial set of predictors for all models included 43 variables, including patient demographics, lifestyle factors, laboratory tests, currently prescribed medications, selected morbidities, and previous emergency admissions. We then added 13 more variables (marital status, prior general practice visits, and 11 additional morbidities), and also enriched all variables by incorporating temporal information whenever possible (e.g., time since first diagnosis). We also varied the prediction windows to 12, 36, 48, and 60 months after baseline and compared model performances. For internal validation, we used 5-fold cross-validation. When the initial set of variables was used, GBC outperformed RF and CPH, with an area under the receiver operating characteristic curve (AUC) of 0.779 (95% CI 0.777, 0.781), compared to 0.752 (95% CI 0.751, 0.753) and 0.740 (95% CI 0.739, 0.741), respectively. In external validation, we observed an AUC of 0.796, 0.736, and 0.736 for GBC, RF, and CPH, respectively. The addition of temporal information improved AUC across all models. In internal validation, the AUC rose to 0.848 (95% CI 0.847, 0.849), 0.825 (95% CI 0.824, 0.826), and 0.805 (95% CI 0.804, 0.806) for GBC, RF, and CPH, respectively, while the AUC in external validation rose to 0.826, 0.810, and 0.788, respectively. This enhancement also resulted in robust predictions for longer time horizons, with AUC values remaining at similar levels across all models. Overall, compared to the baseline reference CPH model, the final GBC model showed a 10.8% higher AUC (0.848 compared to 0.740) for prediction of risk of emergency admission within 24 months. GBC also showed the best calibration throughout the risk spectrum. Despite the wide range of variables included in models, our study was still limited by the number of variables included; inclusion of more variables could have further improved model performances.ConclusionsThe use of machine learning and additio...

Artificial Intelligence in the healthcare of older people

“…ML is more accurate in respect to statistical methods to predict acute kidney injury in older people [19] and hospitalacquired pneumonia in people with high risk, i.e. mentally ill people, treated with neuroleptic medication [20], functional fall risk [21] and other poor outcomes in older adults, including delirium [22] and the overall risk of emergency admission [23]. The described algorithms have a potential to provide a more complete and accurate assessment of effects of ageing and risk for physical illnesses and falls in older people while providing a clinically useful predictive capability for earlier intervention in those patients at greatest risk of developing them.…”

Personalized benefit‐risk assessments combining clinical trial and real‐world data provide further insights into which patients may benefit most from therapy: Demonstration for a new oral antiplatelet therapy

“…We acknowledge that stepwise regression methods have well‐known limitations; however, due to the small difference in the results utilizing this method compared with other modeling approaches, the more parsimonious models selected by the stepwise regression were carried forward in this instance. More advanced data‐drive techniques, including machine‐learning (ML) approaches, may perform better compared with traditional modeling techniques in certain instances, and it is possible this would have allowed us to develop models with greater predictive ability. However, we note that other studies comparing these approaches in databases similar to CPRD (eg, CALIBER) have also found only modest increases in predictive model ability by using ML approaches in the CV therapeutic area .…”

Big Data Analytics and Sensor-Enhanced Activity Management to Improve Effectiveness and Efficiency of Outpatient Medical Rehabilitation

“…A deep learning algorithm was 100% effective at identifying invasive forms of breast cancer from pathology images [11]. ML led to substantially improved discrimination and calibration for predicting the risk of emergency admissions [12]. A ML model could use biometric data monitored in the intensive care unit (ICU) to suggest types of treatments needed for different symptoms [13].…”