Sensory substitution: Closing the gap between basic research and widespread practical visual rehabilitation

Abstract

Sensory substitution devices (SSDs) have come a long way since first developed for visual rehabilitation. They have produced exciting experimental results, and have furthered our understanding of the human brain. Unfortunately, they are still not used for practical visual rehabilitation, and are currently considered as reserved primarily for experiments in controlled settings. Over the past decade, our understanding of the neural mechanisms behind visual restoration has changed as a result of converging evidence, much of which was gathered with SSDs. This evidence suggests that the brain is more than a pure sensory-machine but rather is a highly flexible task-machine, i.e., brain regions can maintain or regain their function in vision even with input from other senses. This complements a recent set of more promising behavioral achievements using SSDs and new promising technologies and tools. All these changes strongly suggest that the time has come to revive the focus on practical visual rehabilitation with SSDs and we chart several key steps in this direction such as training protocols and self-train tools.

VBTones a visual to auditory device for the blind

“…First, audio-visual SSDs are a better choice rather than tactile ones in power consumption and portability issues [6,7]. When compared to the experimented tactile based SSDs, auditory versions score a higher speed and accuracy which was achieved in this study [4,5].…”

“…The prosthesis is substituting vision by audition (PSVA) [2], vOICe [4] and EyeMusic [6,7,1] are the well-known devices in vision to the auditory conversion field which contain a set of camera and speaker as a connection to the environment. Figure 1 is an illustration of this connection.…”

“…In PSVA, each pixel refers to a specific sinusoidal wave frequency and the pixel luminance is assigned to the sound loudness [2]. While, vOICe and EyeMusic encode vertical coordination to individual frequencies [4,6,7]. The image in vOICe is gray scaled, whereas EyeMusic can demonstrate colors by using musical instruments such as Reggae Organ, Rapman's Reed, choir, string and Brass instruments in pentatonic scales.…”

“…The image in vOICe is gray scaled, whereas EyeMusic can demonstrate colors by using musical instruments such as Reggae Organ, Rapman's Reed, choir, string and Brass instruments in pentatonic scales. For its distinguishable intervals and prevention from resonance effect [6,7]. vOICe ceiling frequency is 5000Hz.…”

"I Always Wanted to See the Night Sky"

“…Whilst the development of better training would help [26], the users were more interested in developing intuitive-to-use devices, which are not only efficient to use when needed but also provide aesthetically pleasing experiences. The interviewees also expressed an interest in learning about features of the world typically experienced by vision (e.g.…”