Emergency medicine patient wait time multivariable prediction models: a multicentre derivation and validation study

Abstract

ObjectivePatients, families and community members would like emergency department wait time visibility. This would improve patient journeys through emergency medicine. The study objective was to derive, internally and externally validate machine learning models to predict emergency patient wait times that are applicable to a wide variety of emergency departments.MethodsTwelve emergency departments provided 3 years of retrospective administrative data from Australia (2017–2019). Descriptive and exploratory analyses were undertaken on the datasets. Statistical and machine learning models were developed to predict wait times at each site and were internally and externally validated. Model performance was tested on COVID-19 period data (January to June 2020).ResultsThere were 1 930 609 patient episodes analysed and median site wait times varied from 24 to 54 min. Individual site model prediction median absolute errors varied from±22.6 min (95% CI 22.4 to 22.9) to ±44.0 min (95% CI 43.4 to 44.4). Global model prediction median absolute errors varied from ±33.9 min (95% CI 33.4 to 34.0) to ±43.8 min (95% CI 43.7 to 43.9). Random forest and linear regression models performed the best, rolling average models underestimated wait times. Important variables were triage category, last-k patient average wait time and arrival time. Wait time prediction models are not transferable across hospitals. Models performed well during the COVID-19 lockdown period.ConclusionsElectronic emergency demographic and flow information can be used to approximate emergency patient wait times. A general model is less accurate if applied without site-specific factors.

Visualising Emergency Department Wait Times; Rapid Iterative Testing to Determine Patient Preferences for Displays

“…The model output predicts the time from triage-to-clinician. The accuracy of the model is approximately +/-20-40 minutes depending on the complexity of the ED and whether an ED uses a global or customised prediction model [40].…”

“…Once in an ED, wait estimates are used to meet patient emotional, logistic and physical needs and also assist paramedics whilst patients wait on stretchers [41]. Wait times can be estimated using prediction models, with varied accuracy [2, 37, 40, 42]. Despite the high volume of patients exposed to ED waits, when displays are designed there is little to no input from patients and families who are the end users of the displays.…”

Building artificial intelligence and machine learning models : a primer for emergency physicians

“…In addition, as more open access ML libraries (containing ML algorithms and other related code), such as scikit-learn.org and CRAN-R packages (cran.r-project.org) and even automated ML (AutoML) become freely available, the temptation is to apply ML to a problem when traditional solutions work well or are more suitable. (4)(5)(6) The current clinical status quo should be examined, and any proposed improvement in its performance considered in terms of potential clinically significant patient or system benefit. A priori determination of an ideally patient centred (or shared) minimal acceptable difference in outcome should be made.…”

Understanding and interpreting artificial intelligence, machine learning and deep learning in Emergency Medicine

“…In addition, AI is facilitating the harnessing of new technology suitable for ED applications and research, such as natural language processing,13 radiomics14 and machine vision 15. Large data repositories are being curated and leveraged to explore correlations between patient variables and urgent care outcomes 10 16 17. Examples of recent studies with a reasonable rationale for using AI methods are summarised in box 1.…”