Recognition of a Virtual Scene via Simulated Prosthetic Vision

Abstract

In order to effectively aid the blind with optimal low-resolution vision and visual recovery training, pathfinding and recognition tests were performed using a simulated visual prosthetic scene. Simple and complex virtual scenes were built using 3DMAX and Unity, and pixelated to three different resolutions (32 × 32, 64 × 64, and 128 × 128) for real-time pixel processing. Twenty subjects were recruited to complete the pathfinding and object recognition tasks within the scene. The recognition accuracy and time required were recorded and analyzed after the trials. In the simple simulated prosthetic vision (SPV) scene, when the resolution was increased from 32 × 32 to 48 × 48, the object recognition time decreased from 92.19 ± 6.97 to 43.05 ± 6.08 s, and the recognition accuracy increased from 51.22 ± 8.53 to 85.52 ± 4.93%. Furthermore, the number of collisions decreased from 10.00 ± 2.31 to 3.00 ± 0.68. When the resolution was increased from 48 × 48 to 64 × 64, the object recognition time further decreased from 43.05 ± 6.08 to 19.46 ± 3.71 s, the recognition accuracy increased from 85.52 ± 4.93 to 96.89 ± 2.06%, and the number of collisions decreased from 3.00 ± 0.68 to 1.00 ± 0.29. In complex scenes, the time required to recognize the room type decreased from 115.00 ± 23.02 to 68.25 ± 17.23 s, and object recognition accuracy increased from 65.69 ± 9.61 to 80.42 ± 7.70% when the resolution increased from 48 × 48 to 64 × 64. When the resolution increased from 64 × 64 to 128 × 128, the time required to recognize the room type decreased from 68.25 ± 17.23 to 44.88 ± 9.94 s, and object recognition accuracy increased from 80.42 ± 7.71 to 85.69 ± 7.39%. Therefore, one can conclude that there are correlations between pathfinding and recognition. When the resolution increased, the time required for recognition decreased, the recognition accuracy increased, and the number of collisions decreased. Although the subjects could partially complete the recognition task at a resolution of 32 × 32, the recognition time was too long and recognition accuracy was not good enough to identify simple scenes. Complex scenes required a resolution of at least 48 × 48 for complete recognition. In addition, increasing the resolution shortened the time required to identify the type of room, and improved the recognition accuracy.

Augmented reality navigation system for visual prosthesis

“…Furthermore, the use of virtual environments can help to avoid the experimental error caused by collisions and the use of touch to identify objects indirectly. Virtual environments have been widely used for experimentation in navigation tasks [26,28,69]. Some SPV research such as Vergnieux et al [28] used a virtual indoor environment to investigate the navigation capabilities that could be restored through two different stimulation strategies consisting in a reduction of the environment view to match the number of electrodes and an object recognition algorithm in order to present only recognized elements.…”

Advances in retinal prosthesis systems

“…112,113 Simulation studies suggest that use of such processing algorithms can boost task performance, including face and object recognition and navigation. 114–118…”

Visual cortical prosthesis: an electrical perspective

“…In terms of functional performance, a low-resolution electrodes array would allow wayfinding with a 6x10 array [98]. For complex scene recognition, more electrodes are needed, Zhao and colleagues suggested at least 48×48 electrodes for complete recognition would be necessary [107].…”