Closed-loop control of anesthesia and mean arterial pressure using reinforcement learning

Padmanabhan, Regina; Meskin, Nader; Haddad, Wassim M.

doi:10.1016/j.bspc.2015.05.013

Cited by 77 publications

(13 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sandu et al studied the blood pressure regulation problem in post cardiac surgery patients using RL [21]. Padmanabhan et al resorted to RL for the control of continuous intravenous infusion of propofol for ICU patients by both considering anesthetic effect and regulating the mean arterial pressure to a desired range [8]. Raghu et al proposed an approach to deduce treatment policies for septic patients by using continuous deep RL methods [22], and Weng et al applied RL to learn personalized optimal glycemic treatments for severely ill septic patients [9].…”

Section: Related Workmentioning

confidence: 99%

“…Emerging in recent years as a powerful trend and paradigm in machine learning, reinforcement learning (RL) [1] has achieved tremendous achievements in solving complex sequential decision making problems in various health care domains, including treatment in HIV [2], cancer [3], diabetics [4], anaemia [5], schizophrenia [6], epilepsy [7], anesthesia [8], and sepsis [9], just to name a few. However, all the existing RL applications are grounded on an available reward function, either in a numerical or an qualitative form, to indicate the goal of treatments by clinicians.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units

Liu

Zhao

2019

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background Reinforcement learning (RL) provides a promising technique to solve complex sequential decision making problems in health care domains. To ensure such applications, an explicit reward function encoding domain knowledge should be specified beforehand to indicate the goal of tasks. However, there is usually no explicit information regarding the reward function in medical records. It is then necessary to consider an approach whereby the reward function can be learned from a set of presumably optimal treatment trajectories using retrospective real medical data. This paper applies inverse RL in inferring the reward functions that clinicians have in mind during their decisions on weaning of mechanical ventilation and sedative dosing in Intensive Care Units (ICUs). Methods We model the decision making problem as a Markov Decision Process, and use a batch RL method, Fitted Q Iterations with Gradient Boosting Decision Tree , to learn a suitable ventilator weaning policy from real trajectories in retrospective ICU data. A Bayesian inverse RL method is then applied to infer the latent reward functions in terms of weights in trading off various aspects of evaluation criterion. We then evaluate how the policy learned using the Bayesian inverse RL method matches the policy given by clinicians, as compared to other policies learned with fixed reward functions. Results Results show that the inverse RL method is capable of extracting meaningful indicators for recommending extubation readiness and sedative dosage, indicating that clinicians pay more attention to patients’ physiological stability (e.g., heart rate and respiration rate), rather than oxygenation criteria ( FiO 2 , PEEP and SpO 2 ) which is supported by previous RL methods. Moreover, by discovering the optimal weights, new effective treatment protocols can be suggested. Conclusions Inverse RL is an effective approach to discovering clinicians’ underlying reward functions for designing better treatment protocols in the ventilation weaning and sedative dosing in future ICUs.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units

Liu

Zhao

2019

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

show abstract

“…Finally, reinforcement learning is scripted to sequentially self-correct from environmental feedback (positive or negative) and therefore improve the overall model function without having labelled data (Kaelbling et al, 1996). While the application of reinforcement learning is less prevalent in clinical research compared to supervised and unsupervised learning, the value of reinforcement learning in clinical trial design is highlighted in numerous studies (Padmanabhan et al, 2015;Yauney and Shah, 2018;Ribba et al, 2020). Moreover, deep learning, inspired by the biological neural communication networks in the brain, is a noteworthy subset of ML algorithms for processing data and extracting patterns that are used for decision-making.…”

Section: Types Of Machine Learningmentioning

confidence: 99%

Machine Learning Techniques for Personalised Medicine Approaches in Immune-Mediated Chronic Inflammatory Diseases: Applications and Challenges

Peng

Jury

Dönnes³

et al. 2021

Front. Pharmacol.

View full text Add to dashboard Cite

In the past decade, the emergence of machine learning (ML) applications has led to significant advances towards implementation of personalised medicine approaches for improved health care, due to the exceptional performance of ML models when utilising complex big data. The immune-mediated chronic inflammatory diseases are a group of complex disorders associated with dysregulated immune responses resulting in inflammation affecting various organs and systems. The heterogeneous nature of these diseases poses great challenges for tailored disease management and addressing unmet patient needs. Applying novel ML techniques to the clinical study of chronic inflammatory diseases shows promising results and great potential for precision medicine applications in clinical research and practice. In this review, we highlight the clinical applications of various ML techniques for prediction, diagnosis and prognosis of autoimmune rheumatic diseases, inflammatory bowel disease, autoimmune chronic kidney disease, and multiple sclerosis, as well as ML applications for patient stratification and treatment selection. We highlight the use of ML in drug development, including target identification, validation and drug repurposing, as well as challenges related to data interpretation and validation, and ethical concerns related to the use of artificial intelligence in clinical research.

show abstract

“…Automatisation of complex tasks, for instance, controlling the infusion rate of a general anaesthetic during surgery through the monitoring of physiological measurements [4].…”

Section: Reinforcementmentioning

confidence: 99%

Reality Check: The Limitations of Artificial Intelligence in Clinical Medicine

Jones

Archer

Germain

2021

International Journal of Digital Health

View full text Add to dashboard Cite

Artificial intelligence is poised to transform clinical medicine, yet for successful implementation to occur we must also appreciate its limitations. The heterogeneity of current research, particularly in relation to the use of data, means that results cannot necessarily be extrapolated to a population level. Robust study designs are required to minimise the introduction of bias into artificial intelligence models and generate a strong body of evidence. Identifying the specific areas of healthcare where artificial intelligence can have the greatest impact will be essential in ensuring it has a positive influence on clinical outcomes and patient experience over the coming years.

show abstract

Closed-loop control of anesthesia and mean arterial pressure using reinforcement learning

Cited by 77 publications

References 24 publications

Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units

Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units

Machine Learning Techniques for Personalised Medicine Approaches in Immune-Mediated Chronic Inflammatory Diseases: Applications and Challenges

Reality Check: The Limitations of Artificial Intelligence in Clinical Medicine

Contact Info

Product

Resources

About