SeeWay: Vision-Language Assistive Navigation for the Visually Impaired

Yang, Zongming; Yang, Liang; Kong, Li Ren; Wei, Ailin; Leaman, Jesse; Brooks, Johnell O.; Li, Bing

doi:10.1109/smc53654.2022.9945087

Cited by 6 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Frequency Reference T1 11 articles [46], [106], [51], [97], [109], [27], [54], [110], [82], [55], [114] T2 25 articles [32], [29], [62], [24], [34], [73], [48], [50], [39], [94], [40], [95], [42], [78], [67], [31], [101], [37], [103], [85], [38], [111], [112], [113], [115] T3 18 articles [86], [87], [88], [90], [71], [64], [93], [35], [72], [74], [36], [96], [98], [69], …”

Section: Work Principlesmentioning

confidence: 99%

See 1 more Smart Citation

Intelligent Environments and Assistive Technologies For Assisting Visually Impairment People: A Systematic Literature Review

de Souza,

Francisco,

Tavares

et al. 2023

Preprint

View full text Add to dashboard Cite

Intelligent Environments (IE) refer to fisical spaces imbued with pervasive and seamless intelligence, created to proactively support individuals in their daily routines. Developments in technologies such as the Internet of Things (IoT) and Artificial Intelligence (AI) have taken these environments from theoretical notions to practical realities. Simultaneously, the field of Ambient Assisted Living (AAL) has made significant strides. Evolving from Assistive Technology (AT), AAL represents an application of IE that specifically seeks to enable individuals —especially those with disabilities or the elderly— to lead healthier, more independent, and dignified lives through the assistance of technology integrated within their living environments. The confluence of IE and AAL has led to the rise of innovative solutions aimed at enhancing the lives of individuals with special needs,such as the Visually Impaired People (VIP). This research presents a systematic literature review investigating the utilization of IE, underpinned by AAL principles, in supporting VIPs. Initially, a pool of 14,760 studies was obtained from 9 databases, all published up to December 2022. After applying specific inclusion and exclusion criteria, this pool was reduced to 101 articles. Each of these articles was reviewed, analyzed, and categorized into four functional and operating principle categories to address five research questions. The study proposes multiple taxonomies as an approach to holistically synthesize the various technologies and devices categorized in the reviewed articles. Emerging research challenges and trends in this domain are highlighted, with a substantial trend being the escalating use of Deep Learning (DL) techniques. These techniques have been pivotal in the development of systems focused on object detection, path recognition, and navigation for devices, particularly smartphones, geared towards enhancing the lives of VIPs.

show abstract

Section: Work Principlesmentioning

confidence: 99%

“…Rana et al [118] presented the OPTIC ALLY model that uses a combination of sensors, image processing, and machine learning techniques to provide real-time feedback to visually impaired [73], [50], [39], [101] , [111], [112], [113], [115] [44]…”

Section: Smart Phonesmentioning

confidence: 99%

Intelligent Environments and Assistive Technologies For Assisting Visually Impairment People: A Systematic Literature Review

de Souza,

Francisco,

Tavares

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Navigation guidance: To guide PwVI from point A to point B following a planned path, unactuated devices, such as smartphones and wearables, rely on haptic or audio feedback to give instructions such as going straight and turning right [9], [12], [13], [20]. However, delays and misunderstandings might lead to inevitable deviations, which take time and effort to recover from [10], [12].…”

Section: Related Work a Wayfinding Robots And Technologiesmentioning

confidence: 99%

“…As an alternative, extracting semantic information from ego-centric camera images is much cheaper and easier. For example, SeeWay uses skybox images to represent landmarks [20]. Similarly, Landmark AI offers semantic-related functionalities including describing the environment, reading road signs, and recognizing landmarks using a phone camera [19].…”

Section: Related Work a Wayfinding Robots And Technologiesmentioning

confidence: 99%

See 1 more Smart Citation

Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning

Liu

Chang

Weihang

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Persons with visual impairments (PwVI) have difficulties understanding and navigating spaces around them. Current wayfinding technologies either focus solely on navigation or provide limited communication about the environment. Motivated by recent advances in visual-language grounding and semantic navigation, we propose DRAGON, a guiding robot powered by a dialogue system and the ability to associate the environment with natural language. By understanding the commands from the user, DRAGON is able to guide the user to the desired landmarks on the map, describe the environment, and answer questions from visual observations. Through effective utilization of dialogue, the robot can ground the user's free-form descriptions to landmarks in the environment, and give the user semantic information through spoken language. We conduct a user study with blindfolded participants in an everyday indoor environment. Our results demonstrate that DRAGON is able to communicate with the user smoothly, provide a good guiding experience, and connect users with their surrounding environment in an intuitive manner.

show abstract

Intelligent environments and assistive technologies for assisting visually impaired people: a systematic literature review

Souza,

Francisco,

Rosa Tavares

et al. 2024

Univ Access Inf Soc

View full text Add to dashboard Cite

SeeWay: Vision-Language Assistive Navigation for the Visually Impaired

Cited by 6 publications

References 33 publications

Intelligent Environments and Assistive Technologies For Assisting Visually Impairment People: A Systematic Literature Review

Intelligent Environments and Assistive Technologies For Assisting Visually Impairment People: A Systematic Literature Review

Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning

Intelligent environments and assistive technologies for assisting visually impaired people: a systematic literature review

Contact Info

Product

Resources

About