Optimization of Neuroprosthetic Vision via End-to-end Deep Reinforcement Learning

Küçükoğlu, Burcu; Rueckauer, Bodo; Ahmad, Nasir; Steveninck, Jaap de Ruyter van; Güçlü, Umut; Gerven, Marcel van

doi:10.1101/2022.02.25.482017

Cited by 4 publications

(4 citation statements)

References 28 publications

(48 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, by using controllers based on deep neural networks, we gain access to powerful training tools from deep learning which are beneficial for neural control applications. In practice, the amount of training data will be limited, requiring sample-efficient (Hessel et al, 2018;Küçükoglu et al, 2022a) or few-shot learning (Wang et al, 2020). Plasticity in the neural system or a deterioration of the implant is currently a major limiting factor for neurotechnology (Fernández et al, 2020;Sorrell et al, 2021).…”

Section: Choosing the Learning Methodsmentioning

confidence: 99%

An in-silico framework for modeling optimal control of neural systems

Rueckauer

Gerven

2023

Front. Neurosci.

View full text Add to dashboard Cite

IntroductionBrain-machine interfaces have reached an unprecedented capacity to measure and drive activity in the brain, allowing restoration of impaired sensory, cognitive or motor function. Classical control theory is pushed to its limit when aiming to design control laws that are suitable for large-scale, complex neural systems. This work proposes a scalable, data-driven, unified approach to study brain-machine-environment interaction using established tools from dynamical systems, optimal control theory, and deep learning.MethodsTo unify the methodology, we define the environment, neural system, and prosthesis in terms of differential equations with learnable parameters, which effectively reduce to recurrent neural networks in the discrete-time case. Drawing on tools from optimal control, we describe three ways to train the system: Direct optimization of an objective function, oracle-based learning, and reinforcement learning. These approaches are adapted to different assumptions about knowledge of system equations, linearity, differentiability, and observability.ResultsWe apply the proposed framework to train an in-silico neural system to perform tasks in a linear and a nonlinear environment, namely particle stabilization and pole balancing. After training, this model is perturbed to simulate impairment of sensor and motor function. We show how a prosthetic controller can be trained to restore the behavior of the neural system under increasing levels of perturbation.DiscussionWe expect that the proposed framework will enable rapid and flexible synthesis of control algorithms for neural prostheses that reduce the need for in-vivo testing. We further highlight implications for sparse placement of prosthetic sensor and actuator components.

show abstract

Section: Choosing the Learning Methodsmentioning

confidence: 99%

An in-silico framework for modeling optimal control of neural systems

Rueckauer

Gerven

2023

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…The major advantage of RL is its broad applicability even if the system equations are unknown, nonlinear, not differentiable, or only a part of the states is observed. Another promising feature of using RL is that it enables moving experimental design beyond mechanistic objectives (of neuronal activation) towards behavioral objectives (of reallife tasks) [27]. Finally, by using controllers based on deep neural networks, we gain access to powerful training tools from deep learning which are beneficial for neural control applications.…”

Section: Choosing the Learning Methodsmentioning

confidence: 99%

Optimal Control of Neural Systems

Rueckauer

Gerven

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Brain-machine interfaces have reached an unprecedented capacity to measure and drive activity in the brain, allowing restoration of impaired sensory, cognitive or motor function. Classical control theory is pushed to its limit when aiming to design control laws that are suitable for large-scale, complex neural systems. This work proposes a scalable, data-driven, unified approach to study brain-machine-environment interaction using established tools from dynamical systems, optimal control theory, and deep learning. To unify the methodology, we define the environment, neural system, and prosthesis in terms of differential equations with learnable parameters, which effectively reduce to recurrent neural networks in the discrete-time case. Drawing on tools from optimal control, we describe three ways to train the system: Direct optimization of an objective function, oracle-based learning, and reinforcement learning. These approaches are adapted to different assumptions about knowledge of system equations, linearity, differentiability, and observability. We apply the proposed framework to train an in-silico neural system to perform tasks in a linear and a nonlinear environment, namely particle stabilization and pole balancing. After training, this model is perturbed to simulate impairment of sensor and motor function. We show how a prosthetic controller can be trained to restore the behavior of the neural system under increasing levels of perturbation. We expect that the proposed framework will enable rapid and flexible synthesis of control algorithms for neural prostheses that reduce the need for in-vivo testing. We further highlight implications for sparse placement of prosthetic sensor and actuator components.

show abstract

“…DL techniques have found extensive application in medical and neurological fields such as seizure detection [ 18 ], seizure prediction [ 19 – 21 ], epilepsy diagnosis and classification [ 22 , 23 ], autism [ 24 – 27 ], optimization of neuroprosthetic vision [ 28 ], post-stroke rehabilitation with motor imagery [ 29 ], sentiment analysis [ 30 ], emotion recognition [ 31 , 32 ], patient-specific quality assurance [ 33 ], classification of the intracranial electrocorticogram [ 34 ], brain-computer interface (BCI) for discriminating hand motion planning [ 35 ], dyslexia biomarker detection [ 36 – 38 ], and many other fields such as mobile robots [ 39 ], drone-based water rescue and surveillance [ 40 ], and structural health monitoring in recent years [ 41 – 43 ].…”

Section: Introductionmentioning

confidence: 99%

Multiple Classification of Brain MRI Autism Spectrum Disorder by Age and Gender Using Deep Learning

Nogay,

Adeli

2024

J Med Syst

View full text Add to dashboard Cite

The fact that the rapid and definitive diagnosis of autism cannot be made today and that autism cannot be treated provides an impetus to look into novel technological solutions. To contribute to the resolution of this problem through multiple classifications by considering age and gender factors, in this study, two quadruple and one octal classifications were performed using a deep learning (DL) approach. Gender in one of the four classifications and age groups in the other were considered. In the octal classification, classes were created considering gender and age groups. In addition to the diagnosis of ASD (Autism Spectrum Disorders), another goal of this study is to find out the contribution of gender and age factors to the diagnosis of ASD by making multiple classifications based on age and gender for the first time. Brain structural MRI (sMRI) scans of participators with ASD and TD (Typical Development) were pre-processed in the system originally designed for this purpose. Using the Canny Edge Detection (CED) algorithm, the sMRI image data was cropped in the data pre-processing stage, and the data set was enlarged five times with the data augmentation (DA) techniques. The most optimal convolutional neural network (CNN) models were developed using the grid search optimization (GSO) algorism. The proposed DL prediction system was tested with the five-fold cross-validation technique. Three CNN models were designed to be used in the system. The first of these models is the quadruple classification model created by taking gender into account (model 1), the second is the quadruple classification model created by taking into account age (model 2), and the third is the eightfold classification model created by taking into account both gender and age (model 3). ). The accuracy rates obtained for all three designed models are 80.94, 85.42 and 67.94, respectively. These obtained accuracy rates were compared with pre-trained models by using the transfer learning approach. As a result, it was revealed that age and gender factors were effective in the diagnosis of ASD with the system developed for ASD multiple classifications, and higher accuracy rates were achieved compared to pre-trained models.

show abstract

Optimization of Neuroprosthetic Vision via End-to-end Deep Reinforcement Learning

Cited by 4 publications

References 28 publications

An in-silico framework for modeling optimal control of neural systems

An in-silico framework for modeling optimal control of neural systems

Optimal Control of Neural Systems

Multiple Classification of Brain MRI Autism Spectrum Disorder by Age and Gender Using Deep Learning

Contact Info

Product

Resources

About