Flying a Quadcopter—An Audio Entertainment and Training Game for the Visually Impaired

Abstract

With the increase in the number of sensory substitution devices, the engineering community is confronted with a new challenge: ensuring user training in safe virtual environments before using these devices in real-life situations. We developed a game that uses an original sonification model, which, although not specific to a certain substitution device, can be an effective means of training for orientation in space based on audio stimuli. Thus, the game is not only a means of entertainment for visually impaired (VI) people but also one of training for the use of assistive devices. The game design and audio design are original contributions by the authors. The sonification model, which is crucial for a game dedicated to visually impaired people, is described in detail, both at the user and the implementation level. For better immersion, special sound design techniques have been used, such as ambisonic recordings and impulse response (IR) recordings. The game has been improved gradually, especially the sonification model, based on users’ feedback.

Breaking through Barriers: A Systematic Review of Extended Reality in Education for the Visually Impaired

“…Lastly, Ref. [44] explored an audio game, "Flying a Quadcopter", for sound localization training in individuals with visual impairments, demonstrating the potential for improving spatial skills through auditory cues.…”

“…This platform facilitates instructors in crafting customised virtual environments utilising XML coding. In related research conducted by [44], an innovative audio-based training game titled "Flying a Quadcopter" was examined for its efficacy in teaching spatialized audio cues to VI users. Participants' feedback contributed significantly towards refining both the acoustic fidelity and functionality regarding auditory localization exercises within this application setting, which indicates promising advances in educational methodologies focusing on visual impairment challenges, although it emphasises the necessity for more comprehensive evaluations, particularly those involving multiplayer scenarios with VI participants, before it can be deemed fully effective or ready for broad deployment.…”

“…Technological limitations, such as difficulties with object identification, accuracy, variations among users, and a lack of depth perception in XR systems, contribute to these challenges [16,56]. Gesture recognition accuracy, sound-recording limitations, and the lack of specific data analysis methods for these research data sources also need improvement [44,77,78]. Another technological limitation is related to the development of prototypes, including delays and limitations in creating accessible and engaging prototypes, as well as the scarcity of wellresearched resources and materials [72].…”