Deep Trail-Following Robotic Guide Dog in Pedestrian Environments for People who are Blind and Visually Impaired - Learning from Virtual and Real Worlds

Chuang, Tzu-Kuan; Lin, Ni-Ching; Chen, Jih-Shi; Hung, Chen-Hao; Huang, Yiwei; Tengl, Chunchih; Huang, Haikun; Yu, Lap-Fai; Giarré, L.; Wang, Hsueh‐Cheng

doi:10.1109/icra.2018.8460994

Cited by 57 publications

(39 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although important theoretical and technological advances have occurred for the construction and control of guide robots, state-of-the-art approaches are mainly tailored to the deployment of wheeled vehicles and not legged guide robots (e.g., [2], [3], [4]). Unlike wheeled guide robots, legged robots are inherently unstable complex dynamical systems with hybrid nature and high degrees of freedom (DOF).…”

Section: A Related Workmentioning

confidence: 99%

Hierarchical and Safe Motion Control for Cooperative Locomotion of Robotic Guide Dogs and Humans: A Hybrid Systems Approach

Hamed

Kamidi

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

This paper presents a hierarchical control strategy based on hybrid systems theory, nonlinear control, and safety-critical systems to enable cooperative locomotion of robotic guide dogs and visually impaired people. We address high-dimensional and complex hybrid dynamical models that represent collaborative locomotion. At the high level of the control scheme, local and nonlinear baseline controllers, based on the virtual constraints approach, are designed to induce exponentially stable dynamic gaits. The baseline controller for the leash is assumed to be a nonlinear controller that keeps the human in a safe distance from the dog while following it. At the lower level, a real-time quadratic programming (QP) is solved for modifying the baseline controllers of the robot as well as the leash to avoid obstacles. In particular, the QP framework is set up based on control barrier functions (CBFs) to compute optimal control inputs that guarantee safety while being close to the baseline controllers. The stability of the complex periodic gaits is investigated through the Poincaré return map. To demonstrate the power of the analytical foundation, the control algorithms are transferred into an extensive numerical simulation of a complex model that represents cooperative locomotion of a quadrupedal robot, referred to as Vision 60, and a human model. The complex model has 16 continuoustime domains with 60 state variables and 20 control inputs.

show abstract

Section: A Related Workmentioning

confidence: 99%

Hierarchical and Safe Motion Control for Cooperative Locomotion of Robotic Guide Dogs and Humans: A Hybrid Systems Approach

Hamed

Kamidi

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…Another approach is the adoption of neural network. In [30] two deep Convolutional Neural Network model are presented in order to assist the mobility of blind and visually impaired people. Unfortunately, as described below, this solution is not very efficient in term of time complexity.…”

Section: Related Workmentioning

confidence: 99%

“…Since neural networks are currently receiving a lot of attention in the field of computer vision, we also investigated on the comparison between our proposed measurement functionalities and alternative approaches based on deep-learning. In particular, we considered the solution proposed in [30] for guiding a robotic guide-dog along a pre-defined painted path (identical to our path) by means of a Convolutional Neural Network (CNN). The network works on the images acquired by three different fisheye cameras by classifying the dog movements as aligned to the path, drifted on the left or drifted on the right of the path.…”

Section: Comparison With a Deep Learning Approachmentioning

confidence: 99%

An Indoor and Outdoor Navigation System for Visually Impaired People

et al. 2019

Self Cite

View full text Add to dashboard Cite

In this paper, we present a system that allows visually impaired people to autonomously navigate in an unknown indoor and outdoor environment. The system, explicitly designed for low vision people, can be generalized to other users in an easy way. We assume that special landmarks are posed for helping the users in the localization of pre-defined paths. Our novel approach exploits the use of both the inertial sensors and the camera integrated into the smartphone as sensors. Such a navigation system can also provide direction estimates to the tracking system to the users. The success of out approach is proved both through experimental tests performed in controlled indoor environments and in real outdoor installations. A comparison with deep learning methods has been presented.INDEX TERMS Navigation, Kalman filtering, visually impaired, computer vision, dead reckoning.

show abstract

“…Alongside visual manipulation approaches, haptic feedback can be used to induce changes in walking direction. One example, is the use of robotic guide dogs, who can help to guide the visually impaired safely along paths (Chuang et al, 2018), which may be implemented so that the user holds onto a cane/handle (Chuang et al, 2018), and when the robot turns the user receives haptic feedback based on the mass of the robot indicating movement (Hersh and Johnson, 2010). Another example is galvanic vestibular stimulation in which the placement of electrodes behind an individual's ear, can induce postural adjustments through the use of electrical currents (Maeda et al, 2005).…”

Section: Introductionmentioning

confidence: 99%

The Use of Embedded Context-Sensitive Attractors for Clinical Walking Test Guidance in Virtual Reality

Croucher¹,

Powell²,

Dicks³

et al. 2021

Front. Virtual Real.

View full text Add to dashboard Cite

Virtual reality is increasingly used in rehabilitation and can provide additional motivation when working toward therapeutic goals. However, a particular problem for patients regards their ability to plan routes in unfamiliar environments. Therefore, the aim of this study was to explore how visual cues, namely embedded context-sensitive attractors, can guide attention and walking direction in VR, for clinical walking interventions. This study was designed using a butterfly as the embedded context-sensitive attractor, to guide participant locomotion around the clinical figure of eight walk test, to limit the use of verbal instructions. We investigated the effect of varying the number of attractors for figure of eight path following, and whether there are any negative impacts on perceived autonomy or workload. A total of 24 participants took part in the study and completed six attractor conditions in a counterbalanced order. They also experienced a control VE (no attractors) at the beginning and end of the protocol. Each VE condition lasted a duration of 1 min and manipulated the number of attractors to either singular or multiple alongside, the placement of turning markers (virtual trees) used to represent the cones used in clinical settings for the figure of eight walk test. Results suggested that embedded context-sensitive attractors can be used to guide walking direction, following a figure of eight in VR without impacting perceived autonomy, and workload. However, there appears to be a saturation point, with regards to effectiveness of attractors. Too few objects in a VE may reduce feelings of intrinsic motivation, and too many objects in a VE may reduce the effectiveness of attractors for guiding individuals along a figure of eight path. We conclude by indicating future research directions, for attractors and their use as a guide for walking direction.

show abstract

Deep Trail-Following Robotic Guide Dog in Pedestrian Environments for People who are Blind and Visually Impaired - Learning from Virtual and Real Worlds

Cited by 57 publications

References 15 publications

Hierarchical and Safe Motion Control for Cooperative Locomotion of Robotic Guide Dogs and Humans: A Hybrid Systems Approach

Hierarchical and Safe Motion Control for Cooperative Locomotion of Robotic Guide Dogs and Humans: A Hybrid Systems Approach

An Indoor and Outdoor Navigation System for Visually Impaired People

The Use of Embedded Context-Sensitive Attractors for Clinical Walking Test Guidance in Virtual Reality

Contact Info

Product

Resources

About