Near-optimal Individualized Treatment Recommendations

Abstract

Individualized treatment recommendation (ITR) is an important analytic framework for precision medicine. The goal is to assign proper treatments to patients based on their individual characteristics. From the machine learning perspective, the solution to an ITR problem can be formulated as a weighted classification problem to maximize the average benefit that patients receive from the recommended treatments. Several methods have been proposed for ITR in both binary and multicategory treatment setups. In practice, one may prefer a more flexible recommendation with multiple treatment options. This motivates us to develop methods to obtain a set of near-optimal individualized treatment recommendations alternative to each other, called alternative individualized treatment recommendations (A-ITR). We propose two methods to estimate the optimal A-ITR within the outcome weighted learning (OWL) framework. We show the consistency of these methods and obtain an upper bound for the risk between the theoretically optimal recommendation and the estimated one. We also conduct simulation studies, and apply our methods to a real data set for Type 2 diabetic patients with injectable antidiabetic treatments. These numerical studies have shown the usefulness of the proposed A-ITR framework. We develop a R package aitr which can be found at https://github.com/menghaomiao/aitr.

Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework

“…Recently, a few methods have been developed in the statistics literature on learning the optimal policy in mobile health applications (Ertefaie 2014;Luckett et al 2020;Hu et al 2020;Liao, Qi, and Murphy 2020). In addition, there is a growing literature on adapting reinforcement learning to develop dynamic treatment regimes in precision medicine, to recommend treatment decisions based on individual patients' information (Murphy 2003;Chakraborty, Murphy, and Strecher 2010;Qian and Murphy 2011;Zhao et al 2012;Zhang et al 2013;Song et al 2015;Zhao et al 2015;Zhu et al 2017;Zhang et al 2018;Wang et al 2018;Shi et al 2018aShi et al , 2018bMo, Qi, and Liu 2020;Meng et al 2020).…”

A multiagent reinforcement learning framework for off-policy evaluation in two-sided markets

“…Recently, a number of proposals utilize reinforcement learning in mobile health or two-sided markets (Ertefaie, 2014;Luckett et al, 2019;Chen et al, 2020;Hu et al, 2019;Liao et al, 2020;Wang et al, 2021;Zhou et al, 2021;Li et al, 2022a,b;Liao et al, 2022;Shi et al, 2022a,b). In addition, there is a growing literature on adapting reinforcement learning to develop dynamic treatment regimes in precision medicine, to recommend treatment decisions based on individual patients' information (Murphy, 2003;Chakraborty et al, 2010;Qian and Murphy, 2011;Zhao et al, 2012;Zhang et al, 2013;Song et al, 2015;Zhao et al, 2015;Zhang et al, 2015Zhang et al, , 2018Zhu et al, 2017;Wang et al, 2018;Shi et al, 2018a,b;Mo et al, 2020;Meng et al, 2020;Cai et al, 2021;Fang et al, 2021). All these methods considered a single-agent setup where only one agent exists in the environment.…”