The rehabilitation of blindness, using noninvasive methods, requires sensory substitution. A theoretical frame for sensory substitution has been proposed which consists of a model of the deprived sensory system connected to an inverse model of the substitutive sensory system. This paper addresses the feasibility of this conceptual model in the case of auditory substitution, and its implementation as a rough model of the retina connected to an inverse linear model of the cochlea. We have developed an experimental prototype. It aims at allowing optimization of the sensory substitution process. This prototype is based on a personal computer which is connected to a miniature head-fixed video camera and to headphones. A visual scene is captured. Image processing achieves edge detection and graded resolution. Each picture element (pixel) of the processed image is assigned a sinusoidal tone; weighted summation of these sinewaves builds up a complex auditory signal which is transduced by the headphones. On-line selection of various parameters and real-time functioning of the device allow optimization of parameters during psychophysical experimentations. Assessment of this implementation has been initiated, and has so far demonstrated prototype usefulness for pattern recognition. An integrated circuit of this system is to be developed.
Recognition tasks of simple visual patterns have been used to assess an early visual--auditory sensory-substitution system, consisting of the coupling of a rough model of the human retina with an inverse model of the cochlea, by means of a pixel-frequency relationship. The potential advantage of the device, compared with previous ones, is to give the blind the ability to both localise and recognise visual patterns. Four evaluation sessions assessed the performance of twenty-four blindfolded sighted subjects using the device. Subjects had to recognise twenty-five visual patterns, one at a time, using a head-mounted small camera and interpreting the corresponding sounds given by the device. Half the subjects were trained by means of a correction feedback procedure during ten one-hour training sessions embedded in between the evaluation sessions. Results revealed extremely successful training effects. Performance of trained subjects significantly increased with practice compared with the untrained control group. The improvement was also observed for new patterns, demonstrating a learning-process generalisation. The negative correlation observed between scores and processing time showed that the subjects' response accuracy was related to their speed. In conclusion, simple pattern recognition is possible with a fairly natural vision-to-audition coding scheme, given the possibility for the subjects to have sensory--motor interactions while using the device.
Recognition tasks of simple visual patterns have been used to assess an early visual--auditory sensory-substitution system, consisting of the coupling of a rough model of the human retina with an inverse model of the cochlea, by means of a pixel-frequency relationship. The potential advantage of the device, compared with previous ones, is to give the blind the ability to both localise and recognise visual patterns. Four evaluation sessions assessed the performance of twenty-four blindfolded sighted subjects using the device. Subjects had to recognise twenty-five visual patterns, one at a time, using a head-mounted small camera and interpreting the corresponding sounds given by the device. Half the subjects were trained by means of a correction feedback procedure during ten one-hour training sessions embedded in between the evaluation sessions. Results revealed extremely successful training effects. Performance of trained subjects significantly increased with practice compared with the untrained control group. The improvement was also observed for new patterns, demonstrating a learning-process generalisation. The negative correlation observed between scores and processing time showed that the subjects' response accuracy was related to their speed. In conclusion, simple pattern recognition is possible with a fairly natural vision-to-audition coding scheme, given the possibility for the subjects to have sensory--motor interactions while using the device.
This paper deals with the design of a stereo sound board for rehabilitation of blindness following the principles of sensory substitution. In this method, signals coding the lacking visual information are transmitted to an intact sensory organ. Moreover, real time functioning of the sensory substitution device has to be attempted to favor sensory motor interactions beneficial to the learning process. In the present study, we consider the feasibility of this conceptual model of sensory substitution and its implementation as a rough model of the retina connected to an inverse model of the cochlea.The model of vision is limited to main features of the primary visual system, i.e. lateral inhibition and graded resolution. Lateral inhibition can be stimulated by filtering the initial image with an edge detection filter. Graded resolution is modeled using a multi-resolution artificial retina built on the filtered image. The inverse model of audition achieves a weighted summation of sinusoids (i.e. an inverse Fourier transform) whose amplitudes and phases depend on the corresponding ear, taking into account spectral sensitivity of the human ear. Coupling between the model of vision and the inverse model of audition consists of the association of a specific sinusoid to each pixel of the artificial retina; amplitude of each sinusoid is modulated by the gray level of the corresponding pixel.A specific PC-plugged printed card board has been developed to meet follow the auditory specifications required by the prosthesis: the production of a weighted sum of 256 sinusoids whose phases and frequencies are loaded at card initialization, and whose amplitudes can be modified in real time. This complex sound has to be stereophonic, with different phases and amplitudes for the left and right signals. At this time, no low cost music generator meets these specifications. However, sound processors approach them more and more. Such a processor, the SAM8905 (DREAM s.a. manufacturer), can produce up to 64 monoral sinusoids. Hence, the printed card board contains eight SAM8905's (allowing the generation of 256 binaural sinusoids). The serial outputs of the SAM8905's are converted in parallel by serial-to-parallel shift-registers; then, they are added together. These shift registers and parallel adders are implemented in a FPGA which is followed by two Digital-to-Analog Converters. A control circuitry drives the whole data-path.The conceptual model of sensory substitution of vision by audition has been implemented in a real time functioning experimental prototype, using this stereo sound board. This prototype is based on a Personal Computer which is connected to a miniature head-fixed video camera on the one hand, and to headphones on the other hand. In a second application, called acoustic screen, this sound board is used in an aid device for blind persons using computers with graphical user interface (Macintosh, Windows, etc.). This device transforms into sound an image which is captured on the computer screen, around the mouse.
This paper deals with the design of an experimental sensory substitution prosthesis for rehabilitation of blindness. A conceptual model of sensory substitution is proposed, and the general principles of the prosthesis are described. Psychophysical experimentations based on this conceptual model showed performance increasing during learning.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.