Toward a Fully Automated Artificial Pancreas System Using a Bioinspired Reinforcement Learning Design: In Silico Validation

Lee, Seunghyun; Kim, Jiwon; Park, Sung Woon; Jin, Sang‐Man; Park, Sung Min

doi:10.1109/jbhi.2020.3002022

Cited by 44 publications

(40 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although many pioneering studies have used the UVA/Padova T1D simulator to develop glycemic control algorithms, the different settings in meal-protocols, variability, randomness in the scenarios make it challenging to perform a direct head-to-head comparison between the existing works. In addition, sometimes the existing algorithms are evaluated in combination with basal insulin control [ 15 , 19 ]. Hence, we evaluated the proposed DRL algorithm with commonly employed metrics to comprehensively assess its performance.…”

Section: Discussionmentioning

confidence: 99%

“…RL, a sub-field of machine learning, employs a goal-oriented agent to learn the strategies for sequential decision-making, which has been increasingly applied to glycemic control [ 17 , 18 , 19 ], and in particular, for basal insulin modulation. Deep reinforcement learning (DRL), as a recent breakthrough in machine learning, combines RL with deep learning techniques, achieving the state of art in various high-dimensional and complex tasks, such as the board-game of Go [ 20 ], autonomous driving [ 21 ], and medication dosing [ 22 ].…”

Section: Introductionmentioning

confidence: 99%

“…It also provides a set of virtual T1D subjects, which have been proved to match clinical observations [ 28 ]. Most of the pioneering studies on advanced bolus insulin advisors have used the UVA/Padova T1D simulator to perform in silico tests [ 11 , 13 , 14 , 15 , 19 , 29 ]. A generalized DRL model can be trained in the simulator and then fine-tuned with subject-specific data in a clinical setting.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Insulin Bolus Advisor for Type 1 Diabetes Using Deep Reinforcement Learning

Zhu

Kuang

et al. 2020

Sensors

View full text Add to dashboard Cite

(1) Background: People living with type 1 diabetes (T1D) require self-management to maintain blood glucose (BG) levels in a therapeutic range through the delivery of exogenous insulin. However, due to the various variability, uncertainty and complex glucose dynamics, optimizing the doses of insulin delivery to minimize the risk of hyperglycemia and hypoglycemia is still an open problem. (2) Methods: In this work, we propose a novel insulin bolus advisor which uses deep reinforcement learning (DRL) and continuous glucose monitoring to optimize insulin dosing at mealtime. In particular, an actor-critic model based on deep deterministic policy gradient is designed to compute mealtime insulin doses. The proposed system architecture uses a two-step learning framework, in which a population model is first obtained and then personalized by subject-specific data. Prioritized memory replay is adopted to accelerate the training process in clinical practice. To validate the algorithm, we employ a customized version of the FDA-accepted UVA/Padova T1D simulator to perform in silico trials on 10 adult subjects and 10 adolescent subjects. (3) Results: Compared to a standard bolus calculator as the baseline, the DRL insulin bolus advisor significantly improved the average percentage time in target range (70–180 mg/dL) from 74.1%±8.4% to 80.9%±6.9% (p<0.01) and 54.9%±12.4% to 61.6%±14.1% (p<0.01) in the the adult and adolescent cohorts, respectively, while reducing hypoglycemia. (4) Conclusions: The proposed algorithm has the potential to improve mealtime bolus insulin delivery in people with T1D and is a feasible candidate for future clinical validation.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Insulin Bolus Advisor for Type 1 Diabetes Using Deep Reinforcement Learning

Zhu

Kuang

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…The control algorithm used in the artificial pancreas system has to learn models that are rich enough and adapt to the system as a whole [25]. Particularly, reinforcement learning (RL), a branch of machine learning that is based on interactive learning from an unknown environment [29] has, in recent years, gained increased attention in artificial pancreas research [30][31][32][33][34][35][36][37][38][39]. A complete systematic review of reinforcement learning application in diabetes blood glucose control can be found in [40].…”

Section: Introductionmentioning

confidence: 99%

In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus

et al. 2020

View full text Add to dashboard Cite

In this paper, we test and evaluate policy gradient reinforcement learning for automated blood glucose control in patients with Type 1 Diabetes Mellitus. Recent research has shown that reinforcement learning is a promising approach to accommodate the need for individualized blood glucose level control algorithms. The motivation for using policy gradient algorithms comes from the fact that adaptively administering insulin is an inherently continuous task. Policy gradient algorithms are known to be superior in continuous high-dimensional control tasks. Previously, most of the approaches for automated blood glucose control using reinforcement learning has used a finite set of actions. We use the Trust-Region Policy Optimization algorithm in this work. It represents the state of the art for deep policy gradient algorithms. The experiments are carried out in-silico using the Hovorka model, and stochastic behavior is modeled through simulated carbohydrate counting errors to illustrate the full potential of the framework. Furthermore, we use a model-free approach where no prior information about the patient is given to the algorithm. Our experiments show that the reinforcement learning agent is able to compete with and sometimes outperform state-of-the-art model predictive control in blood glucose regulation.

show abstract

“…The presented simulator reproduces the intra-day glucose variability observed in the clinical data, and also describes the nocturnal glucose increase, and the simulated insulin profiles reflecting real-life data. The FDA's approved UVA/Padova simulator has been tested and improved for fully automated BG control with announced/unannounced meal intake [167]. Roberto et al [168] evaluated the potential benefits and risks of different insulins or dosing schemes to suggest the design of the clinical studies for T1DM.…”

mentioning

confidence: 99%

Artificial Pancreas Control Strategies Used for Type 1 Diabetes Control and Treatment: A Comprehensive Analysis

Mehmood

Ahmad

Arif

et al. 2020

ASI

View full text Add to dashboard Cite

This paper presents a comprehensive survey about the fundamental components of the artificial pancreas (AP) system including insulin administration and delivery, glucose measurement (GM), and control strategies/algorithms used for type 1 diabetes mellitus (T1DM) treatment and control. Our main focus is on the T1DM that emerges due to pancreas’s failure to produce sufficient insulin due to the loss of beta cells (β-cells). We discuss various insulin administration and delivery methods including physiological methods, open-loop, and closed-loop schemes. Furthermore, we report several factors such as hyperglycemia, hypoglycemia, and many other physical factors that need to be considered while infusing insulin in human body via AP systems. We discuss three prominent control algorithms including proportional-integral- derivative (PID), fuzzy logic, and model predictive, which have been clinically evaluated and have all shown promising results. In addition, linear and non-linear insulin infusion control schemes have been formally discussed. To the best of our knowledge, this is the first work which systematically covers recent developments in the AP components with a solid foundation for future studies in the T1DM field.

show abstract

Toward a Fully Automated Artificial Pancreas System Using a Bioinspired Reinforcement Learning Design: In Silico Validation

Cited by 44 publications

References 46 publications

An Insulin Bolus Advisor for Type 1 Diabetes Using Deep Reinforcement Learning

An Insulin Bolus Advisor for Type 1 Diabetes Using Deep Reinforcement Learning

In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus

Artificial Pancreas Control Strategies Used for Type 1 Diabetes Control and Treatment: A Comprehensive Analysis

Contact Info

Product

Resources

About