A Markov decision process approach to optimizing cancer therapy using multiple modalities

Abstract

There are several different modalities, e.g., surgery, chemotherapy, and radiotherapy, that are currently used to treat cancer. It is common practice to use a combination of these modalities to maximize clinical outcomes, which are often measured by a balance between maximizing tumor damage and minimizing normal tissue side effects due to treatment. However, multi-modality treatment policies are mostly empirical in current practice, and are therefore subject to individual clinicians' experiences and intuition. We present a novel formulation of optimal multi-modality cancer management using a finitehorizon Markov decision process approach. Specifically, at each decision epoch, the clinician chooses an optimal treatment modality based on the patient's observed state, which we define as a combination of tumor progression and normal tissue side effect. Treatment modalities are categorized as (1) Type 1, which has a high risk and high reward, but is restricted in the frequency of administration during a treatment course, (2) Type 2, which has a lower risk and lower reward than Type 1, but may be repeated without restriction, and (3) Type 3, no treatment (surveillance), which has the possibility of reducing normal tissue side effect at the risk of worsening tumor progression. Numerical simulations using various intuitive, concave reward functions show the structural insights of optimal policies and demonstrate the potential applications of using a rigorous approach to optimizing multi-modality cancer management.

A Promising Approach to Optimizing Sequential Treatment Decisions for Depression: Markov Decision Process

“…Pap test 3. Colposcopy without Pap test Costs Optimize screening policy No Kim et al, (2009)[ 42 ] Finite (user-defined) Cancer Unclear number of states (defined by OAR and tumor) Choose a non-zero dose in each fraction Patient utility Optimize the treatment decision No Maass and Kim (2020) [ 43 ] Finite (user-defined) Cancer 11 states (defined by history of treatment, tissue side effect, tumor progression) 1. Treatment modalities with a high risk 2.…”

“…Researchers have applied MDP models to optimize initial treatment selection [25,26] and the timing of transplantation [27,28], to compare the effectiveness of different combinations of treatment [29], to optimize screening policy [30], and to prevent disease-related complications [31]. However, 16 studies concern the optimization of treatment decisions [32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47]. Five studies use the MDP to optimize treatment decisions for cancer [30,35,39,42,43], five focus on optimizing the treatment of diabetes mellitus [31][32][33][34]41], and the remaining (N = 13) studies are concerned with liver diseases [27,28], high blood pressure/hypertension [37,40], hepatitis C [44], atherosclerotic cardiovascular disease [45], ischemic heart disease [29,36], atrial fibrillation [38], anemia [47], tuberculosis [46...…”

“…However, 16 studies concern the optimization of treatment decisions [32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47]. Five studies use the MDP to optimize treatment decisions for cancer [30,35,39,42,43], five focus on optimizing the treatment of diabetes mellitus [31][32][33][34]41], and the remaining (N = 13) studies are concerned with liver diseases [27,28], high blood pressure/hypertension [37,40], hepatitis C [44], atherosclerotic cardiovascular disease [45], ischemic heart disease [29,36], atrial fibrillation [38], anemia [47], tuberculosis [46], aneurysms [25], and stroke [26].…”

Steelmaking Process Optimised through a Decision Support System Aided by Self-Learning Machine Learning

“…This concept is used as a mathematical form of the reinforcement learning problem [26]. However, it is used in many other fields, such as medicine [27] and robotics [28], among many others.…”

A Markov Decision Process Framework for Optimal Cancer Chemotherapy Dose Selection

“…They built the model mainly on medical intuition by considering some key features, as well as state-action transition. Maass et al [11] proposed a finite-horizon MDP model for multi-modality cancer treatment based on tumor progression and normal tissue side effect. They categorized the treatment options into three types in terms of tumor reduction, risk to normal tissue, and repeatability.…”