Supervised Optimal Chemotherapy Regimen Based on Offline Reinforcement Learning

Shiranthika, Chamani; Chen, Kuo-Wei; Wang, Chung-Yih; Yang, Chan-Yun; Sudantha, B H; Li, Wei-Fu

doi:10.1109/jbhi.2022.3183854

Cited by 14 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Emerson et al [17] proposed using ORL to learn a safer blood glucose control strategy for people with Type 1 diabetes. Shiranthika et al [18] developed the supervised optimal chemotherapy regimen, which can provide cancer patients with an optimal chemotherapy-dosing schedule, thus assisting oncologists in clinical decision-making. Wang et al [19] used ORL to learn the optimal treatment strategy for sepsis patients in ICU.…”

Section: Offline Reinforcement Learningmentioning

confidence: 99%

Towards Safe Propofol Dosing during General Anesthesia Using Deep Offline Reinforcement Learning

Xiuding¹,

Chen²,

Zhu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Automated anesthesia promises to enable more precise and personalized anesthetic administration and free anesthesiologists from repetitive tasks, allowing them to focus on the most critical aspects of a patient's surgical care. Current research has typically focused on creating simulated environments from which agents can learn. These approaches have demonstrated good experimental results, but are still far from clinical application. In this paper, Policy Constraint Q-Learning (PCQL), a data-driven reinforcement learning algorithm for solving the problem of learning anesthesia strategies on real clinical datasets, is proposed. Conservative Q-Learning was first introduced to alleviate the problem of Q function overestimation in an offline context. A policy constraint term is added to agent training to keep the policy distribution of the agent and the anesthesiologist consistent to ensure safer decisions made by the agent in anesthesia scenarios. The effectiveness of PCQL was validated by extensive experiments on a real clinical anesthesia dataset. Experimental results showthat PCQL is predicted to achieve higher gains than the baseline approach while maintaining good agreement with the reference dose given by the anesthesiologist, using less total dose, and being more responsive to the patient's vital signs. In addition, the confidence intervals of the agent were investigated, which were able to cover most of the clinical decisions of the anesthesiologist. Finally, an interpretable method, SHAP, was used to analyze the contributing components of the model predictions to increase the transparency of the model.

show abstract

Section: Offline Reinforcement Learningmentioning

confidence: 99%

Towards Safe Propofol Dosing during General Anesthesia Using Deep Offline Reinforcement Learning

Xiuding¹,

Chen²,

Zhu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…It's noteworthy that some research endeavors have harnessed offline RL techniques in sensitive domains like personalized patient treatment [30,12]. Intriguingly, as of our knowledge cutoff, there have been no prior attempts to apply offline RL to the conditional de novo drug design.…”

Section: Related Workmentioning

confidence: 99%

Offline RL for generative design of protein binders

Tarasov,

Mbou Sob,

Arbesú

et al. 2023

Preprint

View full text Add to dashboard Cite

Offline Reinforcement Learning (RL) offers a compelling avenue for solving RL problems without the need for interactions with an environment, which may be expensive or unsafe. While online RL methods have found success in various domains, such asde novoStructure-Based Drug Discovery (SBDD), they struggle when it comes to optimizing essential properties derived from protein-ligand docking. The high computational cost associated with the docking process makes it impractical for online RL, which typically requires hundreds of thousands of interactions during learning. In this study, we propose the application of offline RL to address the bottleneck posed by the docking process, leveraging RL’s capability to optimize non-differentiable properties. Our preliminary investigation focuses on using offline RL to conditionally generate drugs with improved docking and chemical properties.

show abstract

“…The identified algorithms primarily aided in the dose individualization of anticoagulants, immunosuppressants and antibiotics [1–5]. In oncology, most of the studies we identified used reinforcement learning, including classical Q-Learning [6–9], deep Q-Learning [10, 11], deep double Q-Learning [12], fuzzy reinforcement learning [13, 14], conservative Q-Learning [15] and other approaches [16, 17]. Recently, several models have been proposed using neural networks for the prediction of drug concentrations [4, 18–20].…”

Section: Introductionmentioning

confidence: 99%

Scientific machine learning for predicting plasma concentrations in anti-cancer therapy

Valderrama,

Teplytska,

Koltermann

et al. 2024

Preprint

View full text Add to dashboard Cite

A variety of classical machine learning approaches have been developed over the past ten years with the aim to individualize drug dosages based on measured plasma concentrations. However, the interpretability of these models is challenging as they do not incorporate information on pharmacokinetic (PK) drug disposition. In this work we compare well-known population PK modelling with classical and a newly proposed scientific machine learning (SciML) framework, which combines knowledge on drug disposition with data-driven modelling. Our approach lets us estimate population PK parameters and their inter-individual variability (IIV) using multimodal covariate data of each patient. A dataset of 549 fluorouracil (5FU) plasma concentrations as example for an intravenously administered drug and a dataset of 308 sunitinib concentrations as example for an orally administered drug were used for analysis. Whereas classical machine learning models were not able to describe the data sufficiently, the proposed model allowed us to obtain highly accurate predictions even for new patients. Additionally, we demonstrated that our model could outperform traditional population PK models in terms of accuracy and greater flexibility when learning population parameters if given enough training data.

show abstract

Supervised Optimal Chemotherapy Regimen Based on Offline Reinforcement Learning

Cited by 14 publications

References 29 publications

Towards Safe Propofol Dosing during General Anesthesia Using Deep Offline Reinforcement Learning

Towards Safe Propofol Dosing during General Anesthesia Using Deep Offline Reinforcement Learning

Offline RL for generative design of protein binders

Scientific machine learning for predicting plasma concentrations in anti-cancer therapy

Contact Info

Product

Resources

About