Model-Based Design of Closed Loop Deep Brain Stimulation Controller using Reinforcement Learning

Gao, Qitong; Naumann, Michael; Jovanov, Ilija; Lesi, Vuk; Kamaravelu, Karthik; Grill, Warren M.; Pajić, Miroslav

doi:10.1109/iccps48487.2020.00018

Cited by 23 publications

(15 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the robustness of the control algorithm was improved at the cost of using a non-standard signal–the unmodulated controller output was applied directly to the stimulated targets, and this may be difficult to implement with an implanted pulse generator. In addition, Gao et al (2020) proposed a deep reinforcement learning-based approach to construct an adaptive DBS framework. The reinforcement signal provided by the environment was an evaluation of the quality of the action that the agent produced.…”

Section: Discussionmentioning

confidence: 99%

Adaptive Parameter Modulation of Deep Brain Stimulation Based on Improved Supervisory Algorithm

Zhu

Wang

et al. 2021

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

Clinically deployed deep brain stimulation (DBS) for the treatment of Parkinson’s disease operates in an open loop with fixed stimulation parameters, and this may result in high energy consumption and suboptimal therapy. The objective of this manuscript is to establish, through simulation in a computational model, a closed-loop control system that can automatically adjust the stimulation parameters to recover normal activity in model neurons. Exaggerated beta band activity is recognized as a hallmark of Parkinson’s disease and beta band activity in model neurons of the globus pallidus internus (GPi) was used as the feedback signal to control DBS of the GPi. Traditional proportional controller and proportional-integral controller were not effective in eliminating the error between the target level of beta power and the beta power under Parkinsonian conditions. To overcome the difficulties in tuning the controller parameters and improve tracking performance in the case of changes in the plant, a supervisory control algorithm was implemented by introducing a Radial Basis Function (RBF) network to build the inverse model of the plant. Simulation results show the successful tracking of target beta power in the presence of changes in Parkinsonian state as well as during dynamic changes in the target level of beta power. Our computational study suggests the feasibility of the RBF network-driven supervisory control algorithm for real-time modulation of DBS parameters for the treatment of Parkinson’s disease.

show abstract

Section: Discussionmentioning

confidence: 99%

Adaptive Parameter Modulation of Deep Brain Stimulation Based on Improved Supervisory Algorithm

Zhu

Wang

et al. 2021

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreover, fixed high-frequency stimulation may have side effects, such as speech impairment. To address the above problems, Gao et al [ 110 ] proposed a deep learning method based on reinforcement learning (RL) [ 111 ] to help derive specific DBS patterns, which were able to provide effective DBS controllers and energy efficiency. This RL-based method was evaluated on a brain-on-a-chip field-programmable gate array (FPGA) [ 112 ] platform to conduct the basal ganglia model (BGM) [ 113 ].…”

Section: Case Studies In Ooc Applicationsmentioning

confidence: 99%

The Synergy between Deep Learning and Organs-on-Chips for High-Throughput Drug Screening: A Review

et al. 2023

View full text Add to dashboard Cite

Organs-on-chips (OoCs) are miniature microfluidic systems that have arguably become a class of advanced in vitro models. Deep learning, as an emerging topic in machine learning, has the ability to extract a hidden statistical relationship from the input data. Recently, these two areas have become integrated to achieve synergy for accelerating drug screening. This review provides a brief description of the basic concepts of deep learning used in OoCs and exemplifies the successful use cases for different types of OoCs. These microfluidic chips are of potential to be assembled as highly potent human-on-chips with complex physiological or pathological functions. Finally, we discuss the future supply with perspectives and potential challenges in terms of combining OoCs and deep learning for image processing and automation designs.

show abstract

“…Therefore, ML can play a guiding role in online adaptive stimulation. 365 , 366 , 367 For instance, the feedback loop can analyze the neural signal’s oscillatory patterns or other reliably detectable biosignals (e.g., biochemcial, electromyographic, and mechanical signals) to classify or detect the critical brain state for delivery of closed-loop neurostimulation. 368 Additionally, reinforcement learning can be applied to learn a state-action value function to identify the best excitability brain state, where the state corresponds to the neural activity (e.g., the amplitude of evoked potentials, characteristics of brain connectivity) and the action corresponds to on/off stimulation mode.…”

Section: Closing the Loop For Testing Causality Through Neurostimulationmentioning

confidence: 99%

“…368 Additionally, reinforcement learning can be applied to learn a state-action value function to identify the best excitability brain state, where the state corresponds to the neural activity (e.g., the amplitude of evoked potentials, characteristics of brain connectivity) and the action corresponds to on/off stimulation mode. 367 , 369 …”

Section: Closing the Loop For Testing Causality Through Neurostimulationmentioning

confidence: 99%

Modern views of machine learning for precision psychiatry

Chen¹,

Kulkarni²,

Galatzer‐Levy³

et al. 2022

Patterns

View full text Add to dashboard Cite

Model-Based Design of Closed Loop Deep Brain Stimulation Controller using Reinforcement Learning

Cited by 23 publications

References 27 publications

Adaptive Parameter Modulation of Deep Brain Stimulation Based on Improved Supervisory Algorithm

Adaptive Parameter Modulation of Deep Brain Stimulation Based on Improved Supervisory Algorithm

The Synergy between Deep Learning and Organs-on-Chips for High-Throughput Drug Screening: A Review

Modern views of machine learning for precision psychiatry

Contact Info

Product

Resources

About