Using Predictive Analytics to Identify Children at High Risk of Defaulting From a Routine Immunization Program: Feasibility Study

Abstract

BackgroundDespite the availability of free routine immunizations in low- and middle-income countries, many children are not completely vaccinated, vaccinated late for age, or drop out from the course of the immunization schedule. Without the technology to model and visualize risk of large datasets, vaccinators and policy makers are unable to identify target groups and individuals at high risk of dropping out; thus default rates remain high, preventing universal immunization coverage. Predictive analytics algorithm leverages artificial intelligence and uses statistical modeling, machine learning, and multidimensional data mining to accurately identify children who are most likely to delay or miss their follow-up immunization visits.ObjectiveThis study aimed to conduct feasibility testing and validation of a predictive analytics algorithm to identify the children who are likely to default on subsequent immunization visits for any vaccine included in the routine immunization schedule.MethodsThe algorithm was developed using 47,554 longitudinal immunization records, which were classified into the training and validation cohorts. Four machine learning models (random forest; recursive partitioning; support vector machines, SVMs; and C-forest) were used to generate the algorithm that predicts the likelihood of each child defaulting from the follow-up immunization visit. The following variables were used in the models as predictors of defaulting: gender of the child, language spoken at the child’s house, place of residence of the child (town or city), enrollment vaccine, timeliness of vaccination, enrolling staff (vaccinator or others), date of birth (accurate or estimated), and age group of the child. The models were encapsulated in the predictive engine, which identified the most appropriate method to use in a given case. Each of the models was assessed in terms of accuracy, precision (positive predictive value), sensitivity, specificity and negative predictive value, and area under the curve (AUC).ResultsOut of 11,889 cases in the validation dataset, the random forest model correctly predicted 8994 cases, yielding 94.9% sensitivity and 54.9% specificity. The C-forest model, SVMs, and recursive partitioning models improved prediction by achieving 352, 376, and 389 correctly predicted cases, respectively, above the predictions made by the random forest model. All models had a C-statistic of 0.750 or above, whereas the highest statistic (AUC 0.791, 95% CI 0.784-0.798) was observed in the recursive partitioning algorithm.ConclusionsThis feasibility study demonstrates that predictive analytics can accurately identify children who are at a higher risk for defaulting on follow-up immunization visits. Correct identification of potential defaulters opens a window for evidence-based targeted interventions in resource limited settings to achieve optimal immunization coverage and timeliness.

Predicting vaccine hesitancy from area‐level indicators: A machine learning approach

“…Overall, our performances are comparable with the ones shown in Chandir et al. (2018) using individual data. More in detail, Chandir et al.…”

“…For instance, Chandir et al. (2018) adopt four machine learning models (random forest [RF], recursive partitioning, support vector machines, and C‐forest) to generate the algorithm that predicts the likelihood of each child defaulting in Pakistan from the follow‐up immunization visit. Bell et al.…”

Artificial intelligence and the future of global health

“…Machine learning classification tools were also used to estimate the risk of non-infectious disease health outcomes. For example, studies have focused on estimating anaemia risk in children using standardised household survey data, 43 identifying children with the greatest risk of missing immunisation sessions, 44 and detecting high-risk births using cardiotocography data. 45 A study from Brazil aimed to assess the behavioural risk classification of sexually active teenagers.…”

Big Data Applications in Vaccinology

“…Some researchers also look at the impact of newspapers on the vaccination decision (Okuhara et al, 2019b). Other researchers examine social media as a tool to promote on-time vaccinations for children (Chandir et al, 2018;Bell et al, 2019).…”

“…Finally, the third sub-theme includes papers that describe certain architectural aspects of systems built specifically for reverse vaccinology research. For example, Dharayani and colleagues (2019) describe a MapReduce-based architecture to run the BLAST Algorithm, an open source tool for Chan, Jamieson & Albarracin, 2020;Du et al, 2017;Dunn et al, 2017;Getman et al, 2018;Kagashe et al, 2017;Krittanawong et al, 2017;Massey et al, 2016;Martin et al, 2020;Meyer et al, 2019;Nawa et al, 2016;Okuhara et al, 2019a,b;Okuhara et al, 2018;Pananos et al, 2017;Tavoschi et al, 2020;Tomeny, Vargo & El-Toukhy, 2017;Walter, Ophir & Jamieson, 2020Bots Bell et al, 2019Chandir et al, 2018;Kudugunta & Ferrara, 2018;Yuan, Schuchard & Crooks, 2019 comparing biological sequence information. Table 5 displays some of the papers included in the Technology of Vaccinology theme.…”