Leveraging Augmented Reality to Create Apps for People with Visual Disabilities

Yoon, Chris; Louie, Ryan; Ryan, Jeremy; Vu, MinhKhang; Bang, Hyegi; Derksen, William; Ruvolo, Paul

doi:10.1145/3308561.3353788

Cited by 54 publications

(22 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A further approach is presented in [ 26 ], where a novel system based on augmented reality and vibrotactile stimulation of the smartphone is proposed. The system uses a virtual target point that changes progressively.…”

Section: Related Workmentioning

confidence: 99%

A Navigation and Augmented Reality System for Visually Impaired People

Valvo

Croce

Garlisi

et al. 2021

Sensors

View full text Add to dashboard Cite

In recent years, we have assisted with an impressive advance in augmented reality systems and computer vision algorithms, based on image processing and artificial intelligence. Thanks to these technologies, mainstream smartphones are able to estimate their own motion in 3D space with high accuracy. In this paper, we exploit such technologies to support the autonomous mobility of people with visual disabilities, identifying pre-defined virtual paths and providing context information, reducing the distance between the digital and real worlds. In particular, we present ARIANNA+, an extension of ARIANNA, a system explicitly designed for visually impaired people for indoor and outdoor localization and navigation. While ARIANNA is based on the assumption that landmarks, such as QR codes, and physical paths (composed of colored tapes, painted lines, or tactile pavings) are deployed in the environment and recognized by the camera of a common smartphone, ARIANNA+ eliminates the need for any physical support thanks to the ARKit library, which we exploit to build a completely virtual path. Moreover, ARIANNA+ adds the possibility for the users to have enhanced interactions with the surrounding environment, through convolutional neural networks (CNNs) trained to recognize objects or buildings and enabling the possibility of accessing contents associated with them. By using a common smartphone as a mediation instrument with the environment, ARIANNA+ leverages augmented reality and machine learning for enhancing physical accessibility. The proposed system allows visually impaired people to easily navigate in indoor and outdoor scenarios simply by loading a previously recorded virtual path and providing automatic guidance along the route, through haptic, speech, and sound feedback.

show abstract

Section: Related Workmentioning

confidence: 99%

A Navigation and Augmented Reality System for Visually Impaired People

Valvo

Croce

Garlisi

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The same technology used in an assisted return system could be used to help a person follow a path previously taken by another individual. An example is given by Clew, an augmented reality app designed for visually impaired users [1]. Clew allows one to record a route, then load it and receive directions in accessible format when the same person (or someone else) wants to traverse the same route.…”

Section: Pedestrian Dead Reckoning (Pdr)mentioning

confidence: 99%

“…In fact, mechanisms similar to assisted return have been proposed for other applications, e.g. to let users make available specific paths in a building for others to follow [1] [2]. Of course, if a map is available, the system could certainly leverage this information.…”

Section: Introductionmentioning

confidence: 99%

Finding Your Way Back: Comparing Path Odometry Algorithms for Assisted Return

Tsai

Ren

Elyasi

et al. 2021

2021 IEEE International Conference on Pervasive Computing and Communications Workshops and Other Affiliated Events (PerCom Work

View full text Add to dashboard Cite

We present a comparative analysis of inertial-based odometry algorithms for the purpose of assisted return. An assisted return system facilitates backtracking of a path previously taken, and can be particularly useful for blind pedestrians. We present a new algorithm for path matching, and test it in simulated assisted return tasks with data from WeAllWalk, the only existing data set with inertial data recorded from blind walkers. We consider two odometry systems, one based on deep learning (RoNIN), and the second based on robust turn detection and step counting. Our results show that the best path matching results are obtained using the turns/steps odometry system.

show abstract

“…Even without a map, pedestrian tracking could be useful for assisted return: In this case, the system can provide spatial information and direction to a blind user who is backtracking their path. [18][19][20][21]. In the second situation (map-assisted), we assume that a map of the place is available in digital form.…”

Section: Introductionmentioning

confidence: 99%

Smartphone-Based Inertial Odometry for Blind Walkers

Ren

Elyasi

Manduchi

2021

Sensors

View full text Add to dashboard Cite

Pedestrian tracking systems implemented in regular smartphones may provide a convenient mechanism for wayfinding and backtracking for people who are blind. However, virtually all existing studies only considered sighted participants, whose gait pattern may be different from that of blind walkers using a long cane or a dog guide. In this contribution, we present a comparative assessment of several algorithms using inertial sensors for pedestrian tracking, as applied to data from WeAllWalk, the only published inertial sensor dataset collected indoors from blind walkers. We consider two situations of interest. In the first situation, a map of the building is not available, in which case we assume that users walk in a network of corridors intersecting at 45° or 90°. We propose a new two-stage turn detector that, combined with an LSTM-based step counter, can robustly reconstruct the path traversed. We compare this with RoNIN, a state-of-the-art algorithm based on deep learning. In the second situation, a map is available, which provides a strong prior on the possible trajectories. For these situations, we experiment with particle filtering, with an additional clustering stage based on mean shift. Our results highlight the importance of training and testing inertial odometry systems for assisted navigation with data from blind walkers.

show abstract

Leveraging Augmented Reality to Create Apps for People with Visual Disabilities

Cited by 54 publications

References 18 publications

A Navigation and Augmented Reality System for Visually Impaired People

A Navigation and Augmented Reality System for Visually Impaired People

Finding Your Way Back: Comparing Path Odometry Algorithms for Assisted Return

Smartphone-Based Inertial Odometry for Blind Walkers

Contact Info

Product

Resources

About