On optimal treatment regimes selection for mean survival time

Logan

2017 IEEE International Conference on Healthcare Informatics (ICHI)

et al. 2017

In this paper, we propose the first deep reinforcement learning framework to estimate the optimal Dynamic Treatment Regimes from observational medical data. This framework is more flexible and adaptive for high dimensional action and state spaces than existing reinforcement learning methods to model real life complexity in heterogeneous disease progression and treatment choices, with the goal to provide doctor and patients the data-driven personalized decision recommendations. The proposed deep reinforcement learning framework contains a supervised learning step to predict the most possible expert actions; and a deep reinforcement learning step to estimate the long term value function of Dynamic Treatment Regimes. We motivated and implemented the proposed framework on a data set from the Center for International Bone Marrow Transplant Research (CIBMTR) registry database, focusing on the sequence of prevention and treatments for acute and chronic graft versus host disease. We showed results of the initial implementation that demonstrates promising accuracy in predicting human expert decisions and initial implementation for the reinforcement learning step.

Section: Discussionmentioning

confidence: 99%

Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data

Logan

2017 IEEE International Conference on Healthcare Informatics (ICHI)

et al. 2017

“…We first consider the procedure proposed in [26], where R is defined as

\frac{normalΔ Y}{{\overset{false^}{S}}_{C} false(Y false| A, Y false)} - true \int {\overset{false^}{E}}_{\tilde{T}} false{T false| T > t, A, X false} {\frac{d N_{C} false(t false)}{{\overset{false^}{S}}_{C} false(t false| A, X false)} + I false(Y_{i} \geq t false) \frac{d {\overset{false^}{S}}_{C} false(t false| A, X false)}{{\overset{false^}{S}}_{C} {false(t false| A, X false)}^{2}}} .

Here,

{\overset{false^}{S}}_{C} false(t false| A, X false)

and

{\overset{false^}{E}}_{\tilde{T}} false(T false| T > t, A, X false)

are estimated from the Cox model for simplicity. We also consider a more direct clinical measurement without the double robustness correction, which can be interpreted in a similar way as the expected survival time or the restricted mean survival time [6, 16, 21]. To be specific, we consider a restricted mean (log) survival time truncated at τ defined as δT + (1 − δ ) E ( T ), and use this as a plug-in quantity of R in the testing performance calculation.…”

Section: Discussionmentioning

confidence: 99%

Tree based weighted learning for estimating individualized treatment rules with censored data

Cui¹,

Zhu²,

Kosorok³

2017

Electron. J. Statist.

Estimating individualized treatment rules is a central task for personalized medicine. [23] and [22] proposed outcome weighted learning to estimate individualized treatment rules directly through maximizing the expected outcome without modeling the response directly. In this paper, we extend the outcome weighted learning to right censored survival data without requiring either inverse probability of censoring weighting or semiparametric modeling of the censoring and failure times as done in [26]. To accomplish this, we take advantage of the tree based approach proposed in [28] to nonparametrically impute the survival time in two different ways. The first approach replaces the reward of each individual by the expected survival time, while in the second approach only the censored observations are imputed by their conditional expected failure times. We establish consistency and convergence rates for both estimators. In simulation studies, our estimators demonstrate improved performance compared to existing methods. We also illustrate the proposed method on a phase III clinical trial of non-small cell lung cancer.

“…On the other hand, we found that the OTR methods performed worse than the RAFT method, in that it often failed to select the optimal treatments for all signatures. It has been reported that the OTR method performed worse when the censoring rate is high (e.g.,40%) 11 . To investigate this, we simulated two additional datasets using the same settings as for scenario 3 with the censoring rates of 15% and 40%, respectively.…”

Section: Figurementioning

confidence: 99%

“…Performance was additionally compared to an adaptive-LASSO penalized regression method recently developed for treatment selection using AFT models. Our simulation study implements this approach (referred to hereafter as OTR) using the R package OT Rselect 11 . Performance for all methods (BPFT, naive, OTR and RAFT approaches) was compared using various sampling models for the outcomes, as well as differing sets of genes.…”

Section: Simulation Studymentioning

confidence: 99%

“…The average numbers of patients assigned to the non-optimal treatments were 37.4 (SD=27) and 59.1 (SD=24.8) for the method of OTR with the censoring rates of 15% and 40%, respectively; while the corresponding numbers were 23.0 (SD=8.2) and 27.5 (SD=9) for the method of RAFT, respectively. The reader should note that the original study of OTR utilized a much large patient cohort (n=2137) 11 , and thus the poor performance of the OTR methods may be due to the relatively small sample size in our study (n=116). We also evaluated the 3-year restricted mean survival time (RMST) for the targeted therapies based on the subgroup of patients who received the predicted optimal treatment as targeted therapies; and in the same manner calculated the 3-year RMST for the non-targeted therapies. Results obtained using BPFT and RAFT are displayed in Table 6 and Table 7, respectively.…”

Section: Figurementioning

confidence: 99%

See 1 more Smart Citation

Integrating genomic signatures for treatment selection with Bayesian predictive failure time models

Hobbs

Stingo

2016

Stat Methods Med Res

Over the past decade, a tremendous amount of resources have been dedicated to the pursuit of developing genomic signatures that effectively match patients with targeted therapies. Although dozens of therapies that target DNA mutations have been developed, the practice of studying single candidate genes has limited our understanding of cancer. Moreover, many studies of multiple-gene signatures have been conducted for the purpose of identifying prognostic risk cohorts, and thus are limited for selecting personalized treatments. Existing statistical methods for treatment selection often model treatment-by-covariate interactions that are difficult to specify, and require prohibitively large patient cohorts. In this article, we describe a Bayesian predictive failure time (BPFT) model for treatment selection that integrates multiple-gene signatures. Our approach relies on a heuristic measure of similarity that determines the extent to which historically treated patients contribute to the outcome prediction of new patients. The similarity measure, which can be obtained from existing clustering methods, imparts robustness to the underlying stochastic data structure, which enhances feasibility in the presence of small samples. Performance of the proposed method is evaluated in simulation studies, and its application is demonstrated through a study of lung squamous cell carcinoma. Our BPFT approach is shown to effectively leverage genomic signatures to match patients to the therapies that are most beneficial for prolonging their survival.