Battling Antibiotic Resistance: Can Machine Learning Improve Prescribing?

Abstract

Antibiotic resistance constitutes a major health threat. Predicting bacterial causes of infections is key to reducing antibiotic misuse, a leading driver of antibiotic resistance. We train a machine learning algorithm on administrative and microbiological laboratory data from Denmark to predict diagnostic test outcomes for urinary tract infections. Based on predictions, we develop policies to improve prescribing in primary care, highlighting the relevance of physician expertise and policy implementation when patient distributions vary over time. The proposed policies delay antibiotic prescriptions for some patients until test results are known and give them instantly to others. We find that machine learning can reduce antibiotic use by 7.42 percent without reducing the number of treated bacterial infections. As Denmark is one of the most conservative countries in terms of antibiotic use, this result is likely to be a lower bound of what can be achieved elsewhere.that results arrive with a delay of several days, corresponding to nearly a complete course of antibiotic treatment. Machine learning predictions promise a so far unavailable instantaneous bacterial risk assessment. We train a machine learning algorithm, a random forest, on high-dimensional, administrative data from Denmark in 2010-2012 to predict the risk of bacterial presence in laboratory test results from patient samples in primary care, the main source of antibiotic prescribing. 3 The outcome variable, an indicator variable taking the value of one when bacteria are isolated in a patient sample, is based on the microbiological test result physicians receive several days after sending in a sample. The covariates in the prediction model include each individual patient's medical outpatient claims histories, past antibiotic prescriptions, past microbiological test results, a rich set of personal characteristics such as gender, age, detailed employment status and type, education, income, civil status and more, as well as the same information on each individuals' household members. We find that machine learning applied to these data is highly capable of predicting realizations of bacterial UTI in out of sample patient test results.The relevant criterion on which our predictions need to be evaluated, however, is whether or not they can be used to improve human expert decision making. For this purpose, we model prescription decisions as a trade-off between the social cost of prescribing, i.e. promoting resistance, and the health benefits of antibiotic treatment. We build on Kleinberg et al. (2018), who use machine learning predicted risk of defendants committing a crime to show the potential improvements of judges' bail decisions. The model allows us to evaluate reassignment of antibiotic treatment based on the algorithm's prediction of risk, that is, delaying prescriptions for low risk patients until test results are available and instantly giving prescriptions to high risk patients. This reassignment is similar in spirit to Currie and MacLeod (2017) who eva...

The Value of Data for Prediction Policy Problems: Evidence from Antibiotic Prescribing

“…Further personal background information expands the set of achievable to a small extent. Consistent with the results in Ribers and Ullrich (2019), our findings in Figure 5 in Appendix C shows that no positive payoff improvements can be achieved without using diagnostic information encoded in physician decisions. All achievable outcomes, even using the combination of all data segments, are located outside of the area in which the change in prescribing is negative and the change in treated bacterial urinary tract infections is positive.…”

“…Because the policy maker's costs and benefits are unknown, we specify prescription rules that guarantee improvements regardless of a policy maker's preferences. Consistent with the findings in Ribers and Ullrich (2019) policy improvements can only be achieved when physician decisions are used by the machine learning algorithm. As for prediction quality we find that basic personal characteristics and health care data provide the largest incremental improvements.…”

“…Due to the pressing global health problem of increasing antibiotic resistance, it is important to identify policies improving the efficiency of antibiotic use. In the same primary care context in Denmark, Ribers and Ullrich (2019) show that machine learning predictions, combined with physician expertise, can lead to significant improvements in antibiotic prescribing for urinary tract infections. In this Section, we first define an objective function reflecting the trade-off between the costs and benefits of antibiotic use.…”

Machine learning to assist clinical decision-making during the COVID-19 pandemic

“…Identification of lowrisk patients can lead to increased utilization of telehealth and virtual care to avoid unnecessary hospital admissions (Dorsey and Topol 2016). Previous ML models have already been developed to reduce avoidable initial admissions (Ngo et al 2019), predict risk of 30-day readmissions (Frizzell et al 2017;Golas et al 2018), and improve pharmaceutical prescriptions (Ribers and Ullrich 2019). Even disease diagnosis can be possible using emerging ML/AI technology; Parkinson's Disease can be successfully detected by a smartphone-based monitoring platform that extracts features from voice, gait, and reaction time data (Zhan et al 2018).…”

EU Merger Policy Predictability Using Random Forests

“…For the random forests, the AUC lies between 0.62 and 0.71 depending on the model, which is much lower than the AUCs I achieve with the random forests (see Section 5). Kleinberg, Lakkaraju, Leskovec, Ludwig, and Mullainathan (2018) and Ribers and Ullrich (2018) go one step further by not only predicting outcomes but also studying whether the machine learning algorithm makes better decisions than humans. Kleinberg, Lakkaraju, Leskovec, Ludwig, and Mullainathan (2018) analyze the problem of predicting risk of defendants' committing a crime in the context of judges' bail decisions using gradient boosted decision trees and judging whether machine prediction can improve judges' bail decisions.…”