Towards Immersive Virtual Reality Simulations of Bionic Vision

Abstract

Figure 1: Virtual and real patients for bionic vision. Top row: Retinal prosthesis patient. A microelectrode array is implanted in the eye to stimulate the retina (left). Light captured by an external camera is transformed into electrical pulses delivered to the retina to evoke visual percepts (middle), which a patient uses to walk towards a door (right). Bottom row: Virtual patient. Anatomical data is used to place a simulated implant on a simulated retina (left). Visual input from a virtual reality (VR) device is used to generate realistic predictions of simulated prosthetic vision (SPV, middle), which a virtual patient uses to walk to a simulated door in VR (inner-right), or a door in the real world captured by the head-mounted display's camera (outer-right). Edges stand out due to the specific preprocessing methods used, but a variety of methods can be tested. Behavioral performance can then be compared between real prosthesis patients, SPV of the real world, and SPV of the virtual world.

A systematic review of extended reality (XR) for understanding and augmenting vision loss

“…This viewing mode has been termed “transformative reality” ( Lui, Browne, Kleeman, Drummond, & Li, 2011 , 2012 ) (as opposed to “altered reality,” which is typically used to describe simulated low-vision approaches; Bao & Engel, 2019 ). This allows sighted participants to “see” through the eyes of the bionic eye user, taking into account their head and/or eye movements as they explore a virtual environment ( Kasowski, Wu, & Beyeler, 2021 ).…”

“…Because the involvement of real bionic eye users remains limited (500 implantees worldwide) and challenging (e.g., constant assistance, increased setup time, travel cost), it is not surprising that most behavioral studies that recruited real prosthesis users were reported with a relatively small sample size (one to five participants). While XR technology in combination with SPV may provide a more cost-effective alternative to prototyping novel augmentation strategies ( Kasowski et al., 2021 ), future studies should consider a more direct comparison between their theoretical predictions and the visual experience reported by real bionic eye users ( Beyeler et al., 2019 ; Erickson-Davis & Korzybska, 2021 ).…”

Towards a Smart Bionic Eye: AI-powered artificial vision for the treatment of incurable blindness

“…If done right, the use of a VR prototype may drastically speed up the development process by testing theoretical predictions in high-throughput experiments, the best of which can be validated and improved upon in an iterative process with the bionic eye recipient in the loop (Kasowski et al, 2021). Based on this premise, researchers have developed various image optimization strategies, and assessed their performance by having sighted observers conduct daily visual tasks under SPV (Dagnelie et al, 2007;Al-Atabany et al, 2010;Li et al, 2018;.…”

Furthering Visual Accessibility with Extended Reality (XR): A Systematic Review

“…This allows sighted participants to "see" through the eyes of the bionic eye user, taking into account their head and/or eye movements as they explore a virtual environment [51]. In the following subsections, we will briefly summarize the findings of recent prosthetic vision studies and highlight some common limitations.…”