The purpose of this research is to develop a sensorial substitution system from vision to audition. The intention is to provide a noninvasive solution for people with visual impairment to compensate their disability using brain plasticity. The "prosthesis" is a signal processing system that analyses the visual scene to identify objects of interest and encode them into sound. The encoding should be based on characteristics of the human auditory system so that the generated sounds provide an overview of the visual scene in front of the patient, enabling him to locate each identified object. This should allow people with visual disabilities to move around more easily, even in cluttered environments. This paper describes the image processing to enhance objects from images and gives an overview of the sound encoding.
Since the TVSS (Tactile Vision Substitution System) developed by Bach-Y-Rita in 1960's, several sensorial substitution systems have been developed. In general, the so-called "sensorial substitution" system transform stimuli characteristic of one sensory modality (for example, vision) into stimuli of another sensory modality (for example, audition). These systems are developed to help handicapped persons. We developed a sensorial substitution system from vision to audition. An artificial neural network is used to identify the important parts in the image. The Virtual Acoustic Space technic is used to generate localizable sounds. A sound is associated to each important parts of the image. The entire real-time system has been implemented on iOS platforms (iPhone/iPad/iPod Touch{trade mark, serif}). We associated our system with transparent digital earplugs. This way the user is aware of the audio scene happening around him. The system has been tested on non-blind persons and the results are presented.
Since the Tactile Vision Substitution System (TVSS) developed by Bach-Y-Rita in 1960’s, several sensorial substitution systems have been developed. In general, the so-called “sensorial substitution” system transform stimuli characteristic of one sensory modality (for example, vision) into stimuli of another sensory modality (for example, audition). These systems are developed to help handicapped persons. We developed a sensorial substitution system from vision to audition. An artificial neural network is used to identify the important parts in the image. The virtual acoustic space technic is used to generate localizable sounds. A sound is associated to each important parts of the image. The entire real-time system has been implemented on iOS platforms (IPhone/IPad/IPod Touch). We associated our system with transparent digital earplugs. This way the user is aware of the audio scene happening around him. The system has been tested on non-blind persons and the results are presented.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.