Treating Epilepsy via Adaptive Neurostimulation: A Reinforcement Learning Approach

Abstract

This paper presents a new methodology for automatically learning an optimal neurostimulation strategy for the treatment of epilepsy. The technical challenge is to automatically modulate neurostimulation parameters, as a function of the observed EEG signal, so as to minimize the frequency and duration of seizures. The methodology leverages recent techniques from the machine learning literature, in particular the reinforcement learning paradigm, to formalize this optimization problem. We present an algorithm which is able to automatically learn an adaptive neurostimulation strategy directly from labeled training data acquired from animal brain tissues. Our results suggest that this methodology can be used to automatically find a stimulation strategy which effectively reduces the incidence of seizures, while also minimizing the amount of stimulation applied. This work highlights the crucial role that modern machine learning techniques can play in the optimization of treatment strategies for patients with chronic disorders such as epilepsy.

Evidence-based modeling of network discharge dynamics during periodic pacing to control epileptiform activity

“…One can easily imagine a large set of testable hypotheses that require designing and implementing neural control systems, be it automated exploration of neural dynamics or treatment of neurological diseases (Sun et al, 2008; Jahangiri et al, 1997; Schiff et al, 1994; Pineau et al, 2009; Schiff and Sauer, 2008). Currently, first-principle approaches, while unlimited in the testable hypotheses that they can inform, are unsuitable for control applications because they do not model the causal relationships in real-world neural networks with the accuracy necessary to make effective control decisions; they also provide no mapping between real-world observations and the models state, making it difficult to query the model in a specific real-world scenario.…”

Adaptive control of epileptiform excitability in an in vitro model of limbic seizures

“…This can be achieved by using software that automatically detects an impending seizure and administering a fixed stimulation protocol designed to terminate the seizure (Cohen-Gadol et al, 2003; Durand and Bikson, 2001; Fountas et al, 2005; Kossoff et al, 2004; Loscher and Schmidt, 2004; Osorio et al, 2005; Theodore and Fisher, 2004). This can also be achieved through more sophisticated feedback control methods (Guez et al, 2008; Pineau et al, 2009). …”

Deep Reinforcement Learning in Medicine

“…RL has been used to develop treatment strategies for epilepsy [8] and lung cancer [9]. An approach based on deep RL was recently proposed for developing treatment strategies based on medical registry data [10].…”