Improving the Audio Game–Playing Performances of People with Visual Impairments through Multimodal Training

Abstract

Introduction: As the number of people with visual impairments is continuously increasing, rehabilitation and engineering researchers have identified the need to design sensory substitution devices that would offer assistance and guidance to the blind users for performing navigational tasks. Auditory and haptic cues have been shown to be an effective approach towards creating a rich spatial representation of the environment, so that they are considered for being included into the development of assistive tools that would enable the people with visual impairments to acquire knowledge of the surrounding space in a way close to the visual based perception of the normally sighted individuals. However, achieving efficiency through a sensory substitution device requires extensive training for the blind users to learn how to process the artificial auditory cues and convert them into spatial information. Methods: Considering all the potential advantages which game based learning can provide, we propose a new method for training sound localization and virtual navigational skills of the people with visual impairments in a 3D audio game with hierarchical levels of difficulty. The training procedure is focused on a multimodal (auditory and haptic) learning approach in which the subjects have been asked to listen to 3D sounds while simultaneously perceiving a series of vibrations on a haptic headband, corresponding to the direction of the sound source in space. Results: The results we obtained in a sound localization experiment with 10 people suffering from visual impairments showed that the proposed training strategy resulted in significant improvements in the auditory performance and navigation skills of the subjects, thus ensuring behavioral gains in the spatial perception of the environment.

Breaking through Barriers: A Systematic Review of Extended Reality in Education for the Visually Impaired

“…We argue that these approaches are practical and achievable due to the use of available resources and the compound effect of training that learners with visual impairments receive in educational contexts and life. A third example is presented in the works [34][35][36][37][38][39]; these works focused on using sound. Ref.…”

“…Ref. [36] researched multimodal training to improve the audio game-playing performance of individuals with visual impairments. Refs.…”

“…General limitations in the research include small sample sizes, which limit the generalizability and statistical power of studies (e.g., Ref. [24,36]). Moreover, the reliance on existing literature may hinder the exploration of new ideas and approaches due to a dependence on past studies.…”

“…Partnering with industry experts is also recommended to ensure technical feasibility, usability, and engaging narratives [19]. Moreover, research discoveries should be applied to produce practical assistive devices for individuals with visual impairments [36]. The creation of tangible tools and handheld devices to facilitate game design by individuals who are blind should also be explored [39,72].…”