Full gaze contingency provides better reading performance than head steering alone in a simulation of prosthetic vision

Paraskevoudi, Nadia; Pezaris, John S.

doi:10.1038/s41598-021-86996-4

Cited by 14 publications

(20 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results presented in Figure 1 illustrate the value of including a visuotopic model based on spread of cortical activation to realistically estimate phosphene locations and size. Some previous studies have used a model of cortical magnification (34, 60) or visuotopic mapping (41, 42) in their phosphene simulations. However, our simulator is the first to incorporate empirical models of the current spread in cortical tissue (24, 25, 28) to simulate the effects of stimulation current on the phosphene size.…”

Section: Discussionmentioning

confidence: 99%

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Grinten¹,

Steveninck²,

Lozano

et al. 2022

Preprint

View full text Add to dashboard Cite

Blindness affects millions of people around the world, and is expected to become increasingly prevalent in the years to come. For some blind individuals, a promising solution to restore a form of vision are cortical visual prostheses, which convert camera input to electrical stimulation of the cortex to bypass part of the impaired visual system. Due to the constrained number of electrodes that can be implanted, the artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') is of limited resolution, and a great portion of the field's research attention is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is the non-invasive functional evaluation in sighted subjects or with computational models by making use of simulated prosthetic vision (SPV) pipelines. Although the SPV literature has provided us with some fundamental insights, an important drawback that researchers and clinicians may encounter is the lack of realism in the simulation of cortical prosthetic vision, which limits the validity for real-life applications. Moreover, none of the existing simulators address the specific practical requirements for the electrical stimulation parameters. In this study, we developed a PyTorch-based, fast and fully differentiable phosphene simulator. Our simulator transforms specific electrode stimulation patterns into biologically plausible representations of the artificial visual percepts that the prosthesis wearer is expected to see. The simulator integrates a wide range of both classical and recent clinical results with neurophysiological evidence in humans and non-human primates. The implemented pipeline includes a model of the retinotopic organisation and cortical magnification of the visual cortex. Moreover, the quantitative effect of stimulation strength, duration, and frequency on phosphene size and brightness as well as the temporal characteristics of phosphenes are incorporated in the simulator. Our results demonstrate the suitability of the simulator for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioural experiments. The modular approach of our work makes it ideal for further integrating new insights in artificial vision as well as for hypothesis testing. In summary, we present an open-source, fully differentiable, biologically plausible phosphene simulator as a tool for computational, clinical and behavioural neuroscientists working on visual neuroprosthetics.

show abstract

Section: Discussionmentioning

confidence: 99%

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Grinten¹,

Steveninck²,

Lozano

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The image of the scene is captured by a head mounted camera, digitalized by a computer, and a sub-sampled low resolution image is presented on the HMDs. In this way, a variety of tasks have been evaluated using SPV and encouraging performance results were reported on reading speed [57][58][59][60], navigation [61][62][63], object recognition [45,64], hand-eye coordination [65] and face recognition [57,66], among others. However, visual acuity tests are the principle quantitative measures used to assess the efficacy and cost effectiveness of procedures designed to improve or restore vision [67,68].…”

Section: Discussionmentioning

confidence: 99%

Assessing visual acuity in visual prostheses through a virtual-reality system

Sanchez-Garcia¹,

Morollon-Ruiz²,

Martínez-Cantín³

et al. 2022

Preprint

View full text Add to dashboard Cite

Objective. Current visual implants still provide very low resolution and limited field of view, thus limiting visual acuity in implanted patients. Developments of new strategies of artificial vision simulation systems by harnessing new advancements in technologies are of upmost priorities for the development of new visual devices. Approach. In this work, we take advantage of virtual-reality software paired with a portable head-mounted display and evaluated the performance of normally sighted participants under simulated prosthetic vision with variable field of view and number of pixels. Our simulated prosthetic vision system allows simple experimentation in order to study the design parameters of future visual prostheses. Ten normally sighted participants volunteered for a visual acuity study. Subjects were required to identify computer-generated Landolt-C gap orientation and different stimulus based on light perception, time-resolution, light location and motion perception commonly used for visual acuity examination in the sighted. Visual acuity scores were recorded across different conditions of number of electrodes and size of field of view. Main results. Our results showed that of all conditions tested, a field of view of 20 • and 1000 phosphenes of resolution proved the best, with a visual acuity of 1.3 logMAR. Furthermore, performance appears to be correlated with phosphene density, but showing a diminishing return when field of view is less than 20 • . Significance. The development of new artificial vision simulation systems can be useful to guide the development of new visual devices and the optimization of field of view and resolution to provide a helpful and valuable visual aid to profoundly or totally blind patients.

show abstract

“…The majority of SPV studies in this category used monitors, VR headsets, and AR glasses to improve performance on recognition tasks, such as identifying faces ( Chang et al., 2012 ; Wang et al., 2014 ; Irons et al., 2017 ), text ( Denis, Jouffrais, Mailhes, & Mace, 2014 ; Paraskevoudi & Pezaris, 2021 ), and objects ( Li, Zeng, et al., 2018 ; Wang et al., 2016 ). For instance, a number of studies ( Chang et al., 2012 ; Wang et al., 2014 ; Irons et al., 2017 ) highlighted through simulations that face caricaturing, where prominent facial features are highlighted or enhanced, can improve face recognition for sighted subjects viewing SPV.…”

Section: Research Areasmentioning

confidence: 99%

A systematic review of extended reality (XR) for understanding and augmenting vision loss

et al. 2023

View full text Add to dashboard Cite

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user’s eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person’s residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.

show abstract

Full gaze contingency provides better reading performance than head steering alone in a simulation of prosthetic vision

Cited by 14 publications

References 72 publications

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Assessing visual acuity in visual prostheses through a virtual-reality system

A systematic review of extended reality (XR) for understanding and augmenting vision loss

Contact Info

Product

Resources

About