Closed-Loop Reinforcement Learning Based Deep Brain Stimulation Using SpikerNet: A Computational Model

Coventry, Brandon S.; Bartlett, Edward E.

doi:10.1109/ner52421.2023.10123797

Cited by 5 publications

(14 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SpikerNet’s ability to achieve desired firing patterns in vivo was tested by sampling from distributions of previously evoked responses to create novel, previously unobserved firing response target states for the recorded unit. We determined SpikerNet was able to find target firing states precisely (Fig 5B, mean-squared error = 3.872) within a limited number of search iterations (Fig 5C) as predicted in our computational studies(34). It should be noted that search dynamics are intrinsically stochastic and unique to a given animal, target response, and algorithm seeding.…”

Section: Resultssupporting

confidence: 70%

“…After observation of spatial selectivity in thalamocortical INS, we sought to control small neural populations through closed-loop feedback. Current adaptive DBS systems used in Parkinson’s disease use relatively simple control algorithms centered around reducing β band biomarker correlates of symptomology using single or dual threshold “thermostatic” control(42, 75, 76) which may interfere with activities such as volitional movement(42) and may potentially occlude oscillatory neural dynamics unrelated to disease(39). Control of smaller populations of neurons relevant to disease with control algorithms that encode subject specific firing dynamics may provide targeted treatment and a reduction in off-target side effects.…”

Section: Resultsmentioning

confidence: 99%

“…We have previously shown in computational models that SpikerNet is able to quickly learn stimulus trajectories to achieve desired firing patterns(34). SpikerNet’s ability to achieve desired firing patterns in vivo was tested by sampling from distributions of previously evoked responses to create novel, previously unobserved firing response target states for the recorded unit.…”

Section: Resultsmentioning

confidence: 99%

“…We show that SpikerNet rapidly finds and fits targeted firing patterns (Fig 5B) with search behavior that suggests the ability to fit a wide range of possible neural firing patterns (Fig 5C). We have previously shown in computational models that SpikerNet is flexible to drastic changes in firing patterns(34) suggesting that SpikerNet can adapt to long term changes in neural environments present in chronic, clinical DBS and can reduce the number of trips to the clinic for stimulator adjustments. We also observed evidence of SpikerNet finding target responses through the duration of a subject’s recording period, during which arousal can significantly change firing responses requiring retuning of stimulus parameters (Fig S13).…”

Section: Discussionmentioning

confidence: 99%

“…As single unit thalamocortical recordings elicited from INS have largely been unexplored, previous knowledge cannot be adequately constructed into a highly informative prior. However, our previous experience in thalamus and cortical recordings(4–10), the central auditory pathway (11–13), and in INS parameter selection(14, 15) gives prior information on potential variances of firing rate in cortex from thalamic stimulation. As such, we chose moderately informative distributions (see section 3) so as to not unduly influence the posterior and let the observed data fully inform the posterior.…”

Section: Supporting Informationmentioning

confidence: 99%

See 4 more Smart Citations

Spatially specific, closed-loop infrared thalamocortical deep brain stimulation

Coventry,

Lawlor,

Bagnati

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep brain stimulation (DBS) is a powerful clinical tool for the treatment of circuitopathy-related neurological and psychiatric diseases and disorders such as Parkinson’s disease and obsessive-compulsive disorder. Electrically-mediated DBS, however, is limited by the spread of stimulus currents into tissue unrelated to disease course and treatment, potentially causing undesirable patient side effects. In this work, we utilize infrared neural stimulation (INS), an optical neuromodulation technique that uses near to mid-infrared light to drive graded excitatory and inhibitory responses in nerves and neurons, to facilitate an optical and spatially constrained DBS paradigm. INS has been shown to provide spatially constrained responses in cortical neurons and, unlike other optical techniques, does not require genetic modification of the neural target. In this study, we show that INS produces graded, biophysically relevant single-unit responses with robust information transfer in thalamocortical circuits. Importantly, we show that cortical spread of activation from thalamic INS produces more spatially constrained response profiles than conventional electrical stimulation. Owing to observed spatial precision of INS, we used deep reinforcement learning for closed-loop control of thalamocortical circuits, creating real-time representations of stimulus-response dynamics while driving cortical neurons to precise firing patterns. Our data suggest that INS can serve as a targeted and dynamic stimulation paradigm for both open and closed-loop DBS.Significance StatementDespite initial clinical successes, electrical deep brain stimulation (DBS) is fraught with off-target current spillover into tissue outside of therapeutic targets, giving rise to patient side effects and the reduction of therapeutic efficacy. In this study, we validate infrared neural stimulation (INS) as a spatially constrained optical DBS paradigm by quantifying dose-response profiles and robust information transfer through INS driven thalamocortical circuits. We show that INS elicits biophysically relevant responses which are spatially constrained compared to conventional electrical stimulation, potentially reducing off-target side effects. Leveraging the spatial specificity of thalamocortical INS, we used deep reinforcement learning to close the loop on thalamocortical INS and showed the ability to drive subject-specific thalamocortical circuits to target response states in real time.

show abstract

Section: Resultssupporting

confidence: 70%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Supporting Informationmentioning

confidence: 99%

See 3 more Smart Citations

Spatially specific, closed-loop infrared thalamocortical deep brain stimulation

Coventry,

Lawlor,

Bagnati

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Characterization and closed-loop control of infrared thalamocortical stimulation produces spatially constrained single-unit responses

Coventry,

Lawlor,

Bagnati

et al. 2024

PNAS Nexus

View full text Add to dashboard Cite

Deep brain stimulation (DBS) is a powerful tool for the treatment of circuitopathy-related neurological and psychiatric diseases and disorders such as Parkinson’s disease and obsessive-compulsive disorder, as well as a critical research tool for perturbing neural circuits and exploring neuroprostheses. Electrically-mediated DBS, however, is limited by the spread of stimulus currents into tissue unrelated to disease course and treatment, potentially causing undesirable patient side effects. In this work, we utilize infrared neural stimulation (INS), an optical neuromodulation technique that uses near to mid-infrared light to drive graded excitatory and inhibitory responses in nerves and neurons, to facilitate an optical and spatially constrained DBS paradigm. INS has been shown to provide spatially constrained responses in cortical neurons and, unlike other optical techniques, does not require genetic modification of the neural target. We show that INS produces graded, biophysically relevant single-unit responses with robust information transfer in rat thalamocortical circuits. Importantly, we show that cortical spread of activation from thalamic INS produces more spatially constrained response profiles than conventional electrical stimulation. Owing to observed spatial precision of INS, we used deep reinforcement learning for closed-loop control of thalamocortical circuits, creating real-time representations of stimulus-response dynamics while driving cortical neurons to precise firing patterns. Our data suggest that INS can serve as a targeted and dynamic stimulation paradigm for both open and closed-loop DBS.

show abstract

Practical Bayesian Inference in Neuroscience: Or How I Learned To Stop Worrying and Embrace the Distribution

Coventry,

Bartlett

2023

Preprint

View full text Add to dashboard Cite

Typical statistical practices in biological sciences have been increasingly called into question due to difficulties in replication of an increasing number of studies, much of which is confounded by the relative difficulty of null significance hypothesis testing designs and interpretation of p-values. Bayesian inference, representing a fundamentally different approach to hypothesis testing, is receiving renewed interest as a potential alternative or complement to traditional null significance hypothesis testing due to its ease of interpretation and explicit declarations of prior assumptions. Bayesian models are more mathematically complex than equivalent frequentist approaches, which have historically limited applications to simplified analysis cases. However, the advent of probability distribution sampling tools with exponential increases in computational power now allows for quick and robust inference under any distribution of data. Here we present a practical tutorial on the use of Bayesian inference in the context of neuroscientific studies. We first start with an intuitive discussion of Bayes’ rule and inference followed by the formulation of Bayesian-based regression and ANOVA models using data from a variety of neuroscientific studies. We show how Bayesian inference leads to easily interpretable analysis of data while providing an open-source toolbox to facilitate the use of Bayesian tools.Significance StatementBayesian inference has received renewed interest as an alternative to null-significance hypothesis testing for its interpretability, ability to encapsulate prior knowledge into current inference, and robust model comparison paradigms. Despite this renewed interest, discussions of Bayesian inference are often obfuscated by undue mathematical complexity and misunderstandings underlying the Bayesian inference process. In this article, we aim to empower neuroscientists to adopt Bayesian statistical inference by providing a practical methodological walkthrough using single and multi-unit recordings from the rodent auditory circuit accompanied by a well-documented and user-friendly toolkit containing regression and ANOVA statistical models commonly encountered in neuroscience.

show abstract

Closed-Loop Reinforcement Learning Based Deep Brain Stimulation Using SpikerNet: A Computational Model

Cited by 5 publications

References 17 publications

Spatially specific, closed-loop infrared thalamocortical deep brain stimulation

Spatially specific, closed-loop infrared thalamocortical deep brain stimulation

Characterization and closed-loop control of infrared thalamocortical stimulation produces spatially constrained single-unit responses

Practical Bayesian Inference in Neuroscience: Or How I Learned To Stop Worrying and Embrace the Distribution

Contact Info

Product

Resources

About