The application of computer vision to visual prosthesis

Wang, Jing; Zhu, Haiyi; Liu, Jianyun; Li, Heng; Han, Yanling; Zhou, Ri-Gui; Zhang, Yun

doi:10.1111/aor.14022

Cited by 22 publications

(15 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, this zone or object is simplified to a low-resolution pattern. It is efficient to improve the quality of stimulation pattern in an implanted multiple electrode array with limited spatial resolution [97,98]. There are two types of saliency detectionbased methods in the above process: region-based and object-based detection.…”

Section: Saliency Detection-based Methodsmentioning

confidence: 99%

Artificial intelligence techniques for retinal prostheses: a comprehensive review and future direction

Wang

Fang

Zou³

et al. 2023

J. Neural Eng.

View full text Add to dashboard Cite

Abstract. Objective. Retinal prostheses are promising devices to restore vision for patients with severe AMD or RP disease. The visual processing mechanism embodied in retinal prostheses play an important role in the restoration effect. Its performance depends on our understanding of the retina’s working mechanism and the evolvement of computer vision models. Recently, remarkable progress has been made in the field of processing algorithm for retinal prostheses where the new discovery of the retina’s working principle and state-of-the-arts computer vision models are combined together. Approach. We investigated the related research on artificial intelligence techniques for retinal prostheses. The processing algorithm in these studies could be attributed to three types: computer vision-related methods, biophysical models, and deep learning models. Main results. In this review, we first illustrate the structure and function of the normal and degenerated retina, then demonstrate the vision rehabilitation mechanism of three representative retinal prostheses. It is necessary to summarize the computational frameworks abstracted from the normal retina. In addition, the development and feature of three types of different processing algorithms are summarized. Finally, we analyze the bottleneck in existing algorithms and propose our prospect about the future directions to improve the restoration effect. Significance. This review systematically summarizes existing processing models for predicting the response of the retina to external stimuli. What’s more, the suggestions for future direction may inspire researchers in this field to design better algorithms for retinal prostheses.

show abstract

Section: Saliency Detection-based Methodsmentioning

confidence: 99%

Artificial intelligence techniques for retinal prostheses: a comprehensive review and future direction

Wang

Fang

Zou³

et al. 2023

J. Neural Eng.

View full text Add to dashboard Cite

show abstract

“…Recent studies in healthy subjects under simulated prosthetic vision have investigated other solutions such as saliency detection and object recognition [184]. Saliency detection extracts a region or object of interest by computing from image features, and it can be divided into three types: feature-based, region-based and object-based.…”

Section: Image Processing and Advanced Encoding Strategiesmentioning

confidence: 99%

Advances in visual prostheses: engineering and biological challenges

Borda

Ghezzi²

2022

Prog. Biomed. Eng.

View full text Add to dashboard Cite

Vision is an extraordinary sense through which we can appreciate the beauty of the world we live in, gain invaluable knowledge and communicate with others using visual expression and arts. On the contrary, blindness is a threatening medical condition disrupting the life of affected people and their families. Therefore, restoring sight is one of the open challenges of our society. Today, the synergistic convergence of science and technology holds the potential to provide blind patients with artificial vision using visual prostheses: a type of implantable medical device able to reactivate visual neurons using electrical stimulation. Although clinical trials showed that vision restoration is still far away, significant technological advances make visual prostheses a valuable solution for blind patients. This review is not only a description of the state-of-the-art. Instead, it provides the reader with an update on recent developments, a critical discussion of the open challenges, and an overview of promising future directions.

show abstract

“…Sparse stimulation patterns are desirable to minimize the energy consumption of the neuroprosthesis and to maintain stimulation intensities that are sustainable by the neural tissue in the long term [5, 6]. Previous research [7] has focused on direct mechanistic image translation from camera footage to stimulation patterns, often using a single kind of feature, such as edges, segmentation maps, or keypoint detection (see e.g. [8, 9, 10]).…”

Section: Introductionmentioning

confidence: 99%

Optimization of Neuroprosthetic Vision via End-to-end Deep Reinforcement Learning

Küçükoğlu

Rueckauer

Ahmad

et al. 2022

Preprint

View full text Add to dashboard Cite

Visual neuroprostheses are a promising approach to restore basic sight in visually impaired people. A major challenge is to condense the sensory information contained in a complex environment into meaningful stimulation patterns at low spatial and temporal resolution. Previous approaches considered task-agnostic feature extractors such as edge detectors or semantic segmentation, which are likely suboptimal for specific tasks in complex dynamic environments. As an alternative approach, we propose to optimize stimulation patterns by end-to-end training of a feature extractor using deep reinforcement learning agents in virtual environments. We present a task-oriented evaluation framework to compare different stimulus generation mechanisms, such as static edge-based and adaptive end-to-end approaches like the one introduced here. Our experiments in Atari games show that stimulation patterns obtained via task-dependent end-to-end optimized reinforcement learning result in equivalent or improved performance compared to fixed feature extractors on high difficulty levels. These findings signify the relevance of adaptive reinforcement learning for neuroprosthetic vision in complex environments.

show abstract

The application of computer vision to visual prosthesis

Cited by 22 publications

References 58 publications

Artificial intelligence techniques for retinal prostheses: a comprehensive review and future direction

Artificial intelligence techniques for retinal prostheses: a comprehensive review and future direction

Advances in visual prostheses: engineering and biological challenges

Optimization of Neuroprosthetic Vision via End-to-end Deep Reinforcement Learning

Contact Info

Product

Resources

About