Optimization of Neuroprosthetic Vision via End-to-End Deep Reinforcement Learning

Küçükoğlu, Burcu; Rueckauer, Bodo; Ahmad, Nasir; Steveninck, Jaap de Ruyter van; Güçlü, Umut; Gerven, Marcel van

doi:10.1142/s0129065722500526

Cited by 15 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The major advantage of RL is its broad applicability even if the system equations are unknown, nonlinear, not differentiable, or only a part of the states is observed. Another promising feature of using RL is that it enables moving experimental design beyond mechanistic objectives (of neuronal activation) toward behavioral objectives (of real-life tasks) (Küçükoğlu et al, 2022b ). Further, by using controllers based on deep neural networks, we gain access to powerful training tools from deep learning which are beneficial for neural control applications.…”

Section: Discussionmentioning

confidence: 99%

An in-silico framework for modeling optimal control of neural systems

Rueckauer

Gerven

2023

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

IntroductionBrain-machine interfaces have reached an unprecedented capacity to measure and drive activity in the brain, allowing restoration of impaired sensory, cognitive or motor function. Classical control theory is pushed to its limit when aiming to design control laws that are suitable for large-scale, complex neural systems. This work proposes a scalable, data-driven, unified approach to study brain-machine-environment interaction using established tools from dynamical systems, optimal control theory, and deep learning.MethodsTo unify the methodology, we define the environment, neural system, and prosthesis in terms of differential equations with learnable parameters, which effectively reduce to recurrent neural networks in the discrete-time case. Drawing on tools from optimal control, we describe three ways to train the system: Direct optimization of an objective function, oracle-based learning, and reinforcement learning. These approaches are adapted to different assumptions about knowledge of system equations, linearity, differentiability, and observability.ResultsWe apply the proposed framework to train an in-silico neural system to perform tasks in a linear and a nonlinear environment, namely particle stabilization and pole balancing. After training, this model is perturbed to simulate impairment of sensor and motor function. We show how a prosthetic controller can be trained to restore the behavior of the neural system under increasing levels of perturbation.DiscussionWe expect that the proposed framework will enable rapid and flexible synthesis of control algorithms for neural prostheses that reduce the need for in-vivo testing. We further highlight implications for sparse placement of prosthetic sensor and actuator components.

show abstract

Section: Discussionmentioning

confidence: 99%

An in-silico framework for modeling optimal control of neural systems

Rueckauer

Gerven

2023

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our experiments exemplify how supervision targets obtained from semantic segmentation data can be adopted to promote task-relevant information in the phosphene representation. Furthermore, in addition to reconstruction of the input or labelled targets, another recent study experimented with different decoding tasks, including more interactive, goal-driven tasks in virtual game environments (17). Although these proof-of-principle results remain to be translated to real-world tasks and environments, they provide a valuable basis for further exploration.…”

Section: Discussionmentioning

confidence: 99%

“…To achieve a functional level of vision, scene-processing is required to condense complex visual information from the surroundings in an intelligible pattern of phosphenes (7–13). Many studies employ a simulated prosthetic vision (SPV) paradigm to non-invasively evaluate the functional quality of the prosthetic vision with the help of sighted subjects (9, 14–16) or through ‘end-to-end’ approaches, using in silico models (7, 12, 17). Although the aforementioned SPV literature has provided us with some fundamental insights, an important drawback is the lack of realism and biological plausibility of the simulations.…”

Section: Introductionmentioning

confidence: 99%

“…The advantage of deep learning-based methods is clear: more intelligent and flexible extraction of useful information in camera input leads to less noise or unimportant information in the low-resolution phosphene representation, allowing for more successful completion of tasks. Some recent studies demonstrated that the simulation of prosthetic vision can even be incorporated directly into the deep learning-based optimization pipeline for end-to-end optimization (7, 12, 17). With end-to-end optimization, the image processing can be tailored to an individual user or specific tasks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Grinten¹,

Steveninck²,

Lozano

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Blindness affects millions of people around the world, and is expected to become increasingly prevalent in the years to come. For some blind individuals, a promising solution to restore a form of vision are cortical visual prostheses, which convert camera input to electrical stimulation of the cortex to bypass part of the impaired visual system. Due to the constrained number of electrodes that can be implanted, the artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') is of limited resolution, and a great portion of the field's research attention is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is the non-invasive functional evaluation in sighted subjects or with computational models by making use of simulated prosthetic vision (SPV) pipelines. Although the SPV literature has provided us with some fundamental insights, an important drawback that researchers and clinicians may encounter is the lack of realism in the simulation of cortical prosthetic vision, which limits the validity for real-life applications. Moreover, none of the existing simulators address the specific practical requirements for the electrical stimulation parameters. In this study, we developed a PyTorch-based, fast and fully differentiable phosphene simulator. Our simulator transforms specific electrode stimulation patterns into biologically plausible representations of the artificial visual percepts that the prosthesis wearer is expected to see. The simulator integrates a wide range of both classical and recent clinical results with neurophysiological evidence in humans and non-human primates. The implemented pipeline includes a model of the retinotopic organisation and cortical magnification of the visual cortex. Moreover, the quantitative effect of stimulation strength, duration, and frequency on phosphene size and brightness as well as the temporal characteristics of phosphenes are incorporated in the simulator. Our results demonstrate the suitability of the simulator for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioural experiments. The modular approach of our work makes it ideal for further integrating new insights in artificial vision as well as for hypothesis testing. In summary, we present an open-source, fully differentiable, biologically plausible phosphene simulator as a tool for computational, clinical and behavioural neuroscientists working on visual neuroprosthetics.

show abstract

“…The major advantage of RL is its broad applicability even if the system equations are unknown, nonlinear, not differentiable, or only a part of the states is observed. Another promising feature of using RL is that it enables moving experimental design beyond mechanistic objectives (of neuronal activation) towards behavioral objectives (of reallife tasks) [27]. Finally, by using controllers based on deep neural networks, we gain access to powerful training tools from deep learning which are beneficial for neural control applications.…”

Section: Discussionmentioning

confidence: 99%

Optimal Control of Neural Systems

Rueckauer

Gerven

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Brain-machine interfaces have reached an unprecedented capacity to measure and drive activity in the brain, allowing restoration of impaired sensory, cognitive or motor function. Classical control theory is pushed to its limit when aiming to design control laws that are suitable for large-scale, complex neural systems. This work proposes a scalable, data-driven, unified approach to study brain-machine-environment interaction using established tools from dynamical systems, optimal control theory, and deep learning. To unify the methodology, we define the environment, neural system, and prosthesis in terms of differential equations with learnable parameters, which effectively reduce to recurrent neural networks in the discrete-time case. Drawing on tools from optimal control, we describe three ways to train the system: Direct optimization of an objective function, oracle-based learning, and reinforcement learning. These approaches are adapted to different assumptions about knowledge of system equations, linearity, differentiability, and observability. We apply the proposed framework to train an in-silico neural system to perform tasks in a linear and a nonlinear environment, namely particle stabilization and pole balancing. After training, this model is perturbed to simulate impairment of sensor and motor function. We show how a prosthetic controller can be trained to restore the behavior of the neural system under increasing levels of perturbation. We expect that the proposed framework will enable rapid and flexible synthesis of control algorithms for neural prostheses that reduce the need for in-vivo testing. We further highlight implications for sparse placement of prosthetic sensor and actuator components.

show abstract

Optimization of Neuroprosthetic Vision via End-to-End Deep Reinforcement Learning

Cited by 15 publications

References 25 publications

An in-silico framework for modeling optimal control of neural systems

An in-silico framework for modeling optimal control of neural systems

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Optimal Control of Neural Systems

Contact Info

Product

Resources

About