An optimized content-aware image retargeting method: toward expanding the perceived visual field of the high-density retinal prosthesis recipients

Li, Heng; Zeng, Yajie; Lu, Zhuofan; Cao, Xiaofei; Su, Xiaofan; Sui, Xiaohong; Wang, Jing; Chai, Xinyu

doi:10.1088/1741-2552/aa966d

Cited by 8 publications

(11 citation statements)

References 78 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This fact results in loss of visual information which affects patient perception. However, there are research groups using computer vision approaches to try to expand the perceptible visual field in implanted patients to provide useful information in the peripheral vision [78,79].…”

Section: Discussionmentioning

confidence: 99%

Semantic and structural image segmentation for prosthetic vision

2020

View full text Add to dashboard Cite

Prosthetic vision is being applied to partially recover the retinal stimulation of visually impaired people. However, the phosphenic images produced by the implants have very limited information bandwidth due to the poor resolution and lack of color or contrast. The ability of object recognition and scene understanding in real environments is severely restricted for prosthetic users. Computer vision can play a key role to overcome the limitations and to optimize the visual information in the prosthetic vision, improving the amount of information that is presented. We present a new approach to build a schematic representation of indoor environments for simulated phosphene images. The proposed method combines a variety of convolutional neural networks for extracting and conveying relevant information about the scene such as structural informative edges of the environment and silhouettes of segmented objects. Experiments were conducted with normal sighted subjects with a Simulated Prosthetic Vision system. The results show good accuracy for object recognition and room identification tasks for indoor scenes using the proposed approach, compared to other image processing methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Semantic and structural image segmentation for prosthetic vision

2020

View full text Add to dashboard Cite

show abstract

“…It can be observed that object borders were available for recipients to recognize objects under prosthesis vision 16,27,28,29 . For extracting the robust edge information, we employed the guided filter algorithm in this study, whose main advantages were manifested in denoising and maintaining edge information 26 …”

Section: Methodsmentioning

confidence: 99%

“…It can be observed that object borders were available for recipients to recognize objects under prosthesis vision. 16,27,28,29 For extracting the robust edge information, we employed the guided filter algorithm in this study, whose main advantages were manifested in denoising and maintaining edge information. 26 Similar to the general filtering process, an output image q can be obtained by operating an input image p with a filter kernel W. Differently, in the guided filtering process, the filter kernel function W is associated with a guidance image I, which guides the output image q to keep consistent texture with it.…”

Section: Guided Filtermentioning

confidence: 99%

“…For example, the Argus II developer employed image magnification algorithms to promote the reading ability of the blind. 12 Other researchers also attempted to improve the abilities of prosthesis recipients for face, scene, and object recognition, as well as for obstacle avoidance using the saliency detection model, 29,41 grabCut segmentation algorithm, 28,42 object retargeting algorithm 16,43 and the neural network method. 44 Therefore, we suggested that the integration of two types of image-processing algorithms (visible-light image-processing and infrared imageprocessing algorithms) in the dual-mode cameras utilized in future retinal prostheses would improve the ability of recipients of prostheses to complete visual tasks.…”

Section: Benefits Of the Proposed Algorithm In Eye-hand Coordination ...mentioning

confidence: 99%

“…To cover a larger retinal area, some researchers attempted to develop large electrode arrays that matched the retinal curvature 14,15 . Others proposed content‐aware image retargeting methods to render as much important information as possible in the small visual field provided by the prostheses 16 . These research attempts are part of the concerted efforts of researchers to address the two Gordian knots in the development of retinal prostheses—low visual acuity and narrow visual field.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An infrared image‐enhancement algorithm in simulated prosthetic vision: Enlarging working environment of future retinal prostheses

Liang

Chen

et al. 2022

Artificial Organs

Self Cite

View full text Add to dashboard Cite

Background Most existing retinal prostheses contain a built‐in visible‐light camera module that captures images of the surrounding environment. Thus, in case of insufficient or lack of visible light, the camera fails to work, and the retinal prostheses enter a dormant or “OFF” state. A simple and effective solution is replacing the visible‐light camera with a dual‐mode camera. The present research aimed to achieve two main purposes: (1) to explore whether the dual‐mode camera in prosthesis recipients works under no visible‐light conditions and (2) to assess its performance. Methods To accomplish these aims, we enrolled subjects in a psychophysical experiment under simulated prosthetic vision conditions. We found that the subjects could complete some simple visual tasks, but the recognition performance under the infrared mode was significantly inferior to that under the visible‐light mode. These results inspired us to develop and propose a feasible infrared image‐enhancement processing algorithm. Another psychophysical experiment was performed to verify the feasibility of the algorithm. Results The obtained results showed that the average efficiency of the subjects completing visual tasks using our enhancement algorithm (0.014 ± 0.001) was significantly higher (p < 0.001) than that of subjects using direct pixelization (0.007 ± 0.001). Conclusions We concluded that a dual‐mode camera could be a feasible solution to improving the performance of retinal prostheses as the camera adapted better to the specific existing ambient light conditions. Dual‐mode cameras combined with this infrared image‐enhancement algorithm could provide a promising direction for the design of future retinal prostheses.

show abstract

Influence of field of view in visual prostheses design: Analysis with a VR system

Sanchez-Garcia¹,

Martínez-Cantín²,

Bermúdez-Cameo³

et al. 2020

J. Neural Eng.

View full text Add to dashboard Cite

Objective. Visual prostheses are designed to restore partial functional vision in patients with total vision loss. Retinal visual prostheses provide limited capabilities as a result of low resolution, limited field of view and poor dynamic range. Understanding the influence of these parameters in the perception results can guide prostheses research and design. Approach. In this work, we evaluate the influence of field of view with respect to spatial resolution in visual prostheses, measuring the accuracy and response time in a search and recognition task. Twenty-four normally sighted participants were asked to find and recognize usual objects, such as furniture and home appliance in indoor room scenes. For the experiment, we use a new simulated prosthetic vision system that allows simple and effective experimentation. Our system uses a virtual-reality environment based on panoramic scenes. The simulator employs a head-mounted display which allows users to feel immersed in the scene by perceiving the entire scene all around. Our experiments use public image datasets and a commercial head-mounted display. We have also released the virtual-reality software for replicating and extending the experimentation. Main results. Results show that the accuracy and response time decrease when the field of view is increased. Furthermore, performance appears to be correlated with the angular resolution, but showing a diminishing return even with a resolution of less than 2.3 phosphenes per degree. Significance. Our results seem to indicate that, for the design of retinal prostheses, it is better to concentrate the phosphenes in a small area, to maximize the angular resolution, even if that implies sacrificing field of view.

show abstract

An optimized content-aware image retargeting method: toward expanding the perceived visual field of the high-density retinal prosthesis recipients

Abstract: The proposed method is beneficial to expand the perceived visual field of prosthesis recipients and improve their object detection and recognition performance. It suggests that our method may provide an effective option for image processing module in future high-density retinal implants.

Cited by 8 publications

References 78 publications

Semantic and structural image segmentation for prosthetic vision

Semantic and structural image segmentation for prosthetic vision

An infrared image‐enhancement algorithm in simulated prosthetic vision: Enlarging working environment of future retinal prostheses

Influence of field of view in visual prostheses design: Analysis with a VR system

Contact Info

Product

Resources

About