Simplification of Visual Rendering in Simulated Prosthetic Vision Facilitates Navigation

Abstract

Visual neuroprostheses are still limited and simulated prosthetic vision (SPV) is used to evaluate potential and forthcoming functionality of these implants. SPV has been used to evaluate the minimum requirement on visual neuroprosthetic characteristics to restore various functions such as reading, objects and face recognition, object grasping, etc. Some of these studies focused on obstacle avoidance but only a few investigated orientation or navigation abilities with prosthetic vision. The resolution of current arrays of electrodes is not sufficient to allow navigation tasks without additional processing of the visual input. In this study, we simulated a low resolution array (15 × 18 electrodes, similar to a forthcoming generation of arrays) and evaluated the navigation abilities restored when visual information was processed with various computer vision algorithms to enhance the visual rendering. Three main visual rendering strategies were compared to a control rendering in a wayfinding task within an unknown environment. The control rendering corresponded to a resizing of the original image onto the electrode array size, according to the average brightness of the pixels. In the first rendering strategy, vision distance was limited to 3, 6, or 9 m, respectively. In the second strategy, the rendering was not based on the brightness of the image pixels, but on the distance between the user and the elements in the field of view. In the last rendering strategy, only the edges of the environments were displayed, similar to a wireframe rendering. All the tested renderings, except the 3 m limitation of the viewing distance, improved navigation performance and decreased cognitive load. Interestingly, the distance‐based and wireframe renderings also improved the cognitive mapping of the unknown environment. These results show that low resolution implants are usable for wayfinding if specific computer vision algorithms are used to select and display appropriate information regarding the environment.

Deep Learning–Based Scene Simplification for Bionic Vision

“…Based on this premise, researchers have developed various image optimization strategies, and assessed their performance by having sighted observers (i.e., virtual patients) conduct daily visual tasks under SPV [1,9,11,21,28,37]. Simulation allows a wide range of computer vision systems to be developed and tested without requiring implanted devices.…”

Semantic and structural image segmentation for prosthetic vision

“…To overcome the limitations of implants, SPV researchers have tried to optimize the image presentation to deliver the effective visual information in daily activities. For example, Vergnieux et al [32] limited the cues in a virtual scene using different renderings methods, highlighting structural cues such as the edges of different surfaces for navigation. For the same purpose, Perez et al [35] proposed a phosphene map coding using a ground representation of the obstacle-free space and a ceiling representation based on vanishing lines.…”

“…We evaluate and compare the proposed semantic and structural image segmentation with baseline methods through a Simulated Prosthetic Vision (SPV) experiment, which is a standard procedure for non-invasive evaluation using normal vision subjects [19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36]. The experiments included two tasks: object recognition and room identification.…”

Artificial Organs 2017: A Year in Review

“…Vergnieux et al of the Université de Toulouse and CNRS, IRIT, UMR5505, Toulouse, France studied simulated prosthetic vision to evaluate the functionality of implanted visual neuroprostheses for navigation using a low resolution array (15 × 18 electrodes). All the tested renderings, except the 3 m limitation of the viewing distance, improved navigation performance and decreased cognitive load.…”