Computational challenges and opportunities for a bi-directional artificial retina

Abstract

A future artificial retina that can restore high acuity vision in blind people will rely on the capability to both read (observe) and write (control) the spiking activity of neurons using an adaptive, bi-directional and high-resolution device. Although current research is focused on overcoming the technical challenges of building and implanting such a device, exploiting its capabilities to achieve more acute visual perception will also require substantial computational advances. Using high-density large-scale recording and stimulation in the primate retina with an ex vivo multi-electrode array lab prototype, we frame several of the major computational problems, and describe current progress and future opportunities in solving them. First, we identify cell types and locations from spontaneous activity in the blind retina, and then efficiently estimate their visual response properties by using a low-dimensional manifold of inter-retina variability learned from a large experimental dataset. Second, we estimate retinal responses to a large collection of relevant electrical stimuli by passing current patterns through an electrode array, spike sorting the resulting recordings and using the results to develop a model of evoked responses. Third, we reproduce the desired responses for a given visual target by temporally dithering a diverse collection of electrical stimuli within the integration time of the visual system. Together, these novel approaches may substantially enhance artificial vision in a next-generation device.

Retinal Ganglion Cells—Diversity of Cell Types and Clinical Relevance

“…The availability of high-throughput transcriptomic techniques that can be conducted at the single cell level is an exciting development, providing us with powerful tools to identify pathways that can be modulated for generalizable, mutation-independent neuroprotective strategies (213). Although appealing, regenerative medicine will require not only the replacement of the missing RGCs, but also the establishment of the sophisticated circuitry that allows the integration of signals from various pathways to achieve a reasonable degree of visual perception (214).…”

Unraveling neural coding of dynamic natural visual scenes via convolutional recurrent neural networks

“…Video analysis is of great interest to data science researchers, not only for neuroscience, but also for other domains of applied vision, including machine vision, neuromorphic computing, and brain-machine interface, where a large chunk of data in the format of videos is analyzed. 2 Analysis of static natural images is relatively easy, 18 while videos span multiple scales in space and time, which raises tremendous difficulties for analyzing the contexts themselves, 74 as well as for characterizing the underlying neural dynamics. 72 We show that movies with different levels of complexity show different behaviors in models.…”

“…To cope with these inputs, it is necessary to develop an explainable neural network model, either for explaining the data of neuroscience, e.g., the neural response to input scenes, 1 or for developing an efficient computational framework for analyzing dynamic visual scenes for artificial vision. 2 The retina, as the first stage of the visual system, encodes visual information from the external environment in both spatial and temporal domains. 1,3 It consists of three layers of neurons, namely, excitatory photoreceptors (input), bipolar cells, and ganglion cells (output), with inhibitory horizontal and amacrine cells THE BIGGER PICTURE Understanding surrounding environments perceived by eyes requires unraveling the computational principle embedded in the neural system.…”

Representing the dynamics of high-dimensional data with non-redundant wavelets

“…for neural function 13,14 but also for designing decoding algorithms to control the devices of brain-machine interface. [15][16][17] Neural activities are represented as spatiotemporal patterns, in which the dimensionalities of both time and space are high. [18][19][20] One strategy is to reduce the dimension by selecting a subset of features, while keeping information as much as possible.…”