The rehabilitation of blindness, using noninvasive methods, requires sensory substitution. A theoretical frame for sensory substitution has been proposed which consists of a model of the deprived sensory system connected to an inverse model of the substitutive sensory system. This paper addresses the feasibility of this conceptual model in the case of auditory substitution, and its implementation as a rough model of the retina connected to an inverse linear model of the cochlea. We have developed an experimental prototype. It aims at allowing optimization of the sensory substitution process. This prototype is based on a personal computer which is connected to a miniature head-fixed video camera and to headphones. A visual scene is captured. Image processing achieves edge detection and graded resolution. Each picture element (pixel) of the processed image is assigned a sinusoidal tone; weighted summation of these sinewaves builds up a complex auditory signal which is transduced by the headphones. On-line selection of various parameters and real-time functioning of the device allow optimization of parameters during psychophysical experimentations. Assessment of this implementation has been initiated, and has so far demonstrated prototype usefulness for pattern recognition. An integrated circuit of this system is to be developed.

No abstract

No abstract

Recognition tasks of simple visual patterns have been used to assess an early visual--auditory sensory-substitution system, consisting of the coupling of a rough model of the human retina with an inverse model of the cochlea, by means of a pixel-frequency relationship. The potential advantage of the device, compared with previous ones, is to give the blind the ability to both localise and recognise visual patterns. Four evaluation sessions assessed the performance of twenty-four blindfolded sighted subjects using the device. Subjects had to recognise twenty-five visual patterns, one at a time, using a head-mounted small camera and interpreting the corresponding sounds given by the device. Half the subjects were trained by means of a correction feedback procedure during ten one-hour training sessions embedded in between the evaluation sessions. Results revealed extremely successful training effects. Performance of trained subjects significantly increased with practice compared with the untrained control group. The improvement was also observed for new patterns, demonstrating a learning-process generalisation. The negative correlation observed between scores and processing time showed that the subjects' response accuracy was related to their speed. In conclusion, simple pattern recognition is possible with a fairly natural vision-to-audition coding scheme, given the possibility for the subjects to have sensory--motor interactions while using the device.

Pattern recognition in a computer environment was investigated in 6 early blind and 6 blindfolded sighted subjects using auditory substitution of vision. Subjects had to scan visual patterns displayed on a PC screen by moving the pen of a graphics tablet, which lead to corresponding displacements of the cursor on the screen. A small screen area centered on the pointer was then translated into sounds according to a visual-auditory transcription code. Subjects were trained to learn this code during 12 one-hour sessions. Performance of both groups signi®cantly increased with practice. This indicates that mental representations of visual patterns can be acquired through the auditory channel, even in the absence of visual experience. Moreover, blind subjects performed signi®cantly better than sighted subjects did. This could be interpreted as a result of partial compensation for their loss of vision. Pattern recognition in a computer environment is thus possible using a fairly natural vision-to-audition coding scheme.