Predictors of Outpatients’ No-Show: Big Data Analytics using Apache Spark

Abstract

Outpatients who fail to attend their appointments have a negative impact on the healthcare outcome. Thus, healthcare organizations facing new opportunities, one of them is to improve the quality of healthcare. The main challenges is predictive analysis using techniques capable of handle the huge data generated. We propose a big data framework for identifying subject outpatients’ no-show via feature engineering and machine learning (MLlib) in the Spark platform. This study evaluates the performance of five machine learning techniques, using the (2,011,813, 19) outpatients’ visits data. Conducting several experiments and using different validation methods, the Gradient Boosting (GB) performed best, resulting in an increase of accuracy and ROC to 79% and 81%, respectively. In addition, we showed that exploring and evaluating the performance of the machine learning models using various evaluation methods is critical as the accuracy of prediction can significantly differ. The aim of this paper is exploring factors that affect no-show rate and can be used to formulate predictions using big data machine learning techniques

Leveraging machine learning to create user‐friendly models to mitigate appointment failure at dental school clinics

“…These models have been used to predict patient no-show appointments in other studies with positive outcomes. 24,[32][33][34] Altogether, at this point, with the relatively recent application of ML in the healthcare field, it can be understood that one single variable is not enough to determine that one model is better than the other one. Moreover, the success of a model also depends on the availability and quality of the dataset.…”

Development of an evidence-based model for predicting patient, provider, and appointment factors that influence no-shows in a rural healthcare system

“…No-shows occur when a patient fails to appear for a scheduled appointment without prior noti cation to the healthcare practitioner; failing to attend outpatient visits negatively in uences healthcare services, especially in clinics serving medically underserved populations (1)(2)(3)(4). Today, one of the most serious issues confronting health institutions is the presence of patients who fail to show up for their appointments (5).…”

“…The model's proposed overbooking approach demonstrated a signi cant reduction of at least 6% in patient waiting, 27% in overtime, and 3% in total costs when compared to other common at-overbooking methods, however the small sample size of pediatrics clinics limits generalizability to predict no-shows in other specialties or in a large rural health provider network. Another study used outpatient visits data to investigate factors that in uence no-show rates and developed a large data framework for identifying subject outpatients' no-show via feature engineering and machine learning (2). After running multiple experiments and employing several validation approaches, including GB, the accuracy and Receiver Operating Characteristic (ROC) curve increased to 79% and 81%, respectively.…”

“…Previous studies have been undertaken to predict appointment no-shows and develop strategies to mitigate their consequences because no-shows have a detrimental in uence on healthcare systems (2,3,5,6,8,9). In 2015, Woodward et al used logistic regression to predict no-shows in HIV-infected patients.…”

“…Using a sample of 8382 computed tomography appointments, other researchers proposed a model to predict the no-show of ambulatory patients to exam appointments at a radiology department of a major Brazilian public hospital (6). These studies emphasized the importance of investigating and evaluating the performance of machine learning models using a variety of evaluation methodologies, and guided the development of strategies to reduce patient no-shows for exam appointments (2,6). However, they did not take into account the context of a large rural health care network.…”