Residual Weighted Learning for Estimating Individualized Treatment Rules

Abstract

Personalized medicine has received increasing attention among statisticians, computer scientists, and clinical practitioners. A major component of personalized medicine is the estimation of individualized treatment rules (ITRs). Recently, Zhao et al. (2012) proposed outcome weighted learning (OWL) to construct ITRs that directly optimize the clinical outcome. Although OWL opens the door to introducing machine learning techniques to optimal treatment regimes, it still has some problems in performance. (1) The estimated ITR of OWL is affected by a simple shift of the outcome. (2) The rule from OWL tries to keep treatment assignments that subjects actually received. (3) There is no variable selection mechanism with OWL. All of them weaken the finite sample performance of OWL. In this article, we propose a general framework, called Residual Weighted Learning (RWL), to alleviate these problems, and hence to improve finite sample performance. Unlike OWL which weights misclassification errors by clinical outcomes, RWL weights these errors by residuals of the outcome from a regression fit on clinical covariates excluding treatment assignment. We utilize the smoothed ramp loss function in RWL, and provide a difference of convex (d.c.) algorithm to solve the corresponding non-convex optimization problem. By estimating residuals with linear models or generalized linear models, RWL can effectively deal with different types of outcomes, such as continuous, binary and count outcomes. We also propose variable selection methods for linear and nonlinear rules, respectively, to further improve the performance. We show that the resulting estimator of the treatment rule is consistent. We further obtain a rate of convergence for the difference between the expected outcome using the estimated ITR and that of the optimal treatment rule. The performance of the proposed RWL methods is illustrated in simulation studies and in an analysis of cystic fibrosis clinical trial data.

Personalized Dose Finding Using Outcome Weighted Learning

“…However, analogous to the phenomenon illustrated in Zhou et al (2015) for personalized binary treatment, the choice of g ( X ) will impact the performance of O-learning especially when the sample size is small. Choosing a function that can minimize the variance for the dose finding problem would be of great interest.…”

“…Choosing a function that can minimize the variance for the dose finding problem would be of great interest. Zhou et al (2015) recommended a choice of g ( X ) such that R + g ( X ) can be interpreted as the residual of a regression model. The authors argued that the residual better reflected the net treatment benefit than the original outcome.…”

Comparing cluster-level dynamic treatment regimens using sequential, multiple assignment, randomized trials: Regression estimation and sample size considerations

“…49–55 Much of this literature has focused on identifying optimal DTRs at the individual level. Such methods could be extended for the analysis of data arising from cluster-randomized SMARTs to develop optimal cluster-level DTRs.…”

Flexible functional regression methods for estimating individualized treatment rules

“…Qian and Murphy (), Lu et al () and Tian et al () have proposed regression‐based methods in which the response of interest is modelled as a function of treatment assignment and pretreatment covariates. Zhang et al (), Zhang et al (), Zhao et al () and Zhou et al () have proposed policy search methods that seek out the best ITR within a large class of possible rules. Recently, Laber and Zhao () proposed a method for estimating easily interpretable ITRs based on decision trees.…”