Machine Learning–Based Prediction of Growth in Confirmed COVID-19 Infection Cases in 114 Countries Using Metrics of Nonpharmaceutical Interventions and Cultural Dimensions: Model Development and Validation (Preprint)

Abstract

BACKGROUND National governments worldwide have implemented nonpharmaceutical interventions to control the COVID-19 pandemic and mitigate its effects. OBJECTIVE The aim of this study was to investigate the prediction of future daily national confirmed COVID-19 infection growth—the percentage change in total cumulative cases—across 14 days for 114 countries using nonpharmaceutical intervention metrics and cultural dimension metrics, which are indicative of specific national sociocultural norms. METHODS We combined the Oxford COVID-19 Government Response Tracker data set, Hofstede cultural dimensions, and daily reported COVID-19 infection case numbers to train and evaluate five non–time series machine learning models in predicting confirmed infection growth. We used three validation methods—in-distribution, out-of-distribution, and country-based cross-validation—for the evaluation, each of which was applicable to a different use case of the models. RESULTS Our results demonstrate high <i>R</i><sup>2</sup> values between the labels and predictions for the in-distribution method (0.959) and moderate <i>R</i><sup>2</sup> values for the out-of-distribution and country-based cross-validation methods (0.513 and 0.574, respectively) using random forest and adaptive boosting (AdaBoost) regression. Although these models may be used to predict confirmed infection growth, the differing accuracies obtained from the three tasks suggest a strong influence of the use case. CONCLUSIONS This work provides new considerations in using machine learning techniques with nonpharmaceutical interventions and cultural dimensions as metrics to predict the national growth of confirmed COVID-19 infections.

Measuring Association Between Labels and Free-Text Rationales

“…Human simulatability (Doshi-Velez and Kim, 2017) has a rich history in machine learning interpretability as a reliable measure of rationale quality from the lens of utility to an end-user (Kim et al, 2016;Chandrasekaran et al, 2018;Yeung et al, 2020;Poursabzi-Sangdeh et al, 2021;Rajagopal et al, 2021, i.a.). Rather than computing word-level overlap with a ground-truth explanation, simulatability measures the additional predictive ability towards the predicted label a rationale provides over the input, computed as the difference between task performance when a rationale is given as input vs. when it is not (IR→Ô minus I→Ô).…”