A Haptic-Audio Simulator Indoor Navigation: To Assist Visually Impaired Environment Exploration

Abstract

Abstract-VirtuNav is a haptic-, audio-enabled Virtual Reality (VR) simulator that facilitates persons with visual impairment to explore a 3D computerized model of a real-life indoor location, such as a classroom or hospital. For administrative purposes, the screen displays a 2D overhead view of the map to monitor user progress and location relative to the reconstructed 3D environment. The system offers two unique interfaces: a free-roam interface where a user can freely navigate and interact with the model, and an edit mode where an administrator can manage test users, manage maps and retrieve test data. VirtuNav is developed as a practical application offering several unique features including map design, semi-automatic 3D map reconstruction and object classification from 2D map data. Visual and haptic rendering of real-time 3D map navigation are provided, and automated administrative functions including determination of shortest path taken, comparison with the actual path taken, and assessment of performance indicators relating to time taken for exploration and collision data. VirtuNav is a research tool for investigation of user familiarity developed after repeated exposure to the indoor location, to determine the extent to which haptic and/or sound cues improve a visually impaired user's ability to navigate a room or building with or without occlusion. System testing reveals that spatial awareness and memory mapping improve with user iterations within VirtuNav. The application is for greater real world engagement: to build confidence in real world experiences, enabling persons with sight impairment to more comfortably and readily explore and interact with environments formerly unfamiliar or unattainable to them.

Hungry Cat—A Serious Game for Conveying Spatial Information to the Visually Impaired

“…Related studies that involved virtual reality software to provide spatial information to the visually impaired include the Audio Haptic Maze (AHM) [11], the BlindAid [12,13], an auditory-based architectural exploration tool for corridors [14], a training tool for the newly blind [15], a traffic crossing simulator [16], an audio haptic-based exploration of virtual maps using a tactile mouse device [17], an audio haptic-based simulator using haptic device [18], VirtualLeap and VirtualWalk app [19], and a multi-sensory (audio and haptic) virtual environment [20]. Some of these examples use multimodal interfaces, utilizing both audio and haptic feedback [11][12][13]17,18,20]. These were shown to be useful to the visually impaired in creating a better perception of their virtual environment.…”

“…Although the number of tools described in the related studies discussed above were effective at conveying spatial information, there were several drawbacks. Firstly, they were not built with the children in mind [7][8][9][10]12,[14][15][16][17][18][19][20]26], therefore they included complex environments and controls that may be difficult to grasp and can be daunting for younger users. By requiring high levels of cognition and attention spans, the existing spatial information conveying tools do not seem suitable for visually impaired children.…”

“…These systems were mostly developed for use on a desktop computer and many of them require extra equipment to be set up [11][12][13]15,17,18,20], such as the use of orientation and mobility sensors as well as haptic devices. This results in a lack of portability.…”

“…Most of these tools are also not in the form of games, which may be dull and less attractive for children [7][8][9][10][12][13][14][15][16][17][18][19][20]. The formal tone and the lack of interesting elements may bore the children, resulting in a lack of motivation to participate and explore the virtual environment.…”

Loud and Clear: The VR Game Without Visuals

“…Another source of feedback for blind people comes from haptics. While this is a promising area of research to assist the blind in navigating virtual environments [6,20], it poses various constraints (e.g. on resolution, mobility and portability), which might be considered distracting to the player.…”