Enhancing emergency department patient arrival forecasting: a study using feature engineering and advanced machine learning algorithms

Abstract

Background Emergency department (ED) overcrowding is an important problem in many countries. Accurate predictions of patient arrivals in EDs can serve as a management baseline for better allocation of staff and medical resources. In this article, we investigate the use of calendar and meteorological predictors, as well as feature engineered variables, to forecast daily patient arrivals using datasets from eleven different EDs across 3 countries. Methods Six machine learning algorithms were tested, considering forecasting horizons of 7 and 45 days ahead. Tuning of hyperparameters was performed using a grid-search with cross-validation. Algorithms' performance was evaluated using 5-fold cross-validation and four performance metrics. Results The eXtreme Gradient Boosting (XGBoost) achieved better performance considering the two prediction horizons compared to other models, also outperforming results reported in past studies on ED arrival prediction. This is also the first study to utilize Light Gradient Boosting Machine (LightGBM), Support Vector Machine with Radial Basis Function (SVM-RBF) and Neural Network Autoregression (NNAR) for predicting patient arrivals at EDs. Conclusion The Random Forest (RF) variable selection and grid-search methods improved the accuracy of the algorithms tested. Our study innovates by using feature engineering to predict patient arrivals in EDs.