Augmented Reality Powers a Cognitive Prosthesis for the Blind

Abstract

A non-invasive prosthesis for blind people endows objects in the environment with voices, 9 allowing a user to explore the scene, localize objects, and navigate through a building with 10 minimal training. 12 To restore vision for the blind several prosthetic approaches have been explored that convey raw 13 images to the brain. So far these schemes all suffer from a lack of bandwidth and the extensive 14 training required to interpret unusual stimuli. Here we present an alternate approach that restores 15 vision at the cognitive level, bypassing the need to convey sensory data. A wearable computer 16 captures video and other data, extracts the important scene knowledge, and conveys that through 17 auditory augmented reality. This system supports many aspects of visual cognition: from 18 obstacle avoidance to formation and recall of spatial memories, to long-range navigation. Neither 19 training nor modification of the physical environment are required: Blind subjects can navigate 20 an unfamiliar multi-story building on their first attempt. The combination of unprecedented 21 2 computing power in wearable devices with augmented reality technology promises a new era of 22 non-invasive prostheses that are limited only by software. 23 11 Abstract

Virtual Object Replacement Based on Real Environments: Potential Application in Augmented Reality Systems

“…This has led to the AR [2] combines virtual objects within a scene of the real world, thereby removing the primary limitation of VR and eliminating the need to simulate every entity within the scene. This has led to the development of numerous AR applications, such as navigation systems for the blind [3], devices to assist in component assembly [4], and methods to facilitate surgical procedures [5]. The fact that current AR technologies simply place a number of virtual objects within a real environment greatly limits the depth of the simulation.…”