LidSonic V2.0: A LiDAR and Deep-Learning-Based Green Assistive Edge Device to Enhance Mobility for the Visually Impaired

Abstract

Over a billion people around the world are disabled, among whom 253 million are visually impaired or blind, and this number is greatly increasing due to ageing, chronic diseases, and poor environments and health. Despite many proposals, the current devices and systems lack maturity and do not completely fulfill user requirements and satisfaction. Increased research activity in this field is required in order to encourage the development, commercialization, and widespread acceptance of low-cost and affordable assistive technologies for visual impairment and other disabilities. This paper proposes a novel approach using a LiDAR with a servo motor and an ultrasonic sensor to collect data and predict objects using deep learning for environment perception and navigation. We adopted this approach using a pair of smart glasses, called LidSonic V2.0, to enable the identification of obstacles for the visually impaired. The LidSonic system consists of an Arduino Uno edge computing device integrated into the smart glasses and a smartphone app that transmits data via Bluetooth. Arduino gathers data, operates the sensors on the smart glasses, detects obstacles using simple data processing, and provides buzzer feedback to visually impaired users. The smartphone application collects data from Arduino, detects and classifies items in the spatial environment, and gives spoken feedback to the user on the detected objects. In comparison to image-processing-based glasses, LidSonic uses far less processing time and energy to classify obstacles using simple LiDAR data, according to several integer measurements. We comprehensively describe the proposed system’s hardware and software design, having constructed their prototype implementations and tested them in real-world environments. Using the open platforms, WEKA and TensorFlow, the entire LidSonic system is built with affordable off-the-shelf sensors and a microcontroller board costing less than USD 80. Essentially, we provide designs of an inexpensive, miniature green device that can be built into, or mounted on, any pair of glasses or even a wheelchair to help the visually impaired. Our approach enables faster inference and decision-making using relatively low energy with smaller data sizes, as well as faster communications for edge, fog, and cloud computing.

A Novel Algorithm of Distance Calculation Based-on Grid-Edge-Depth-Map and Gyroscope for Visually-Impaired

“…A lower value means a block pixel from the left and right images is close to being similar. The GED map size of the block is 25 pixels [10]. The depth map was built based on a disparity map by calculating the camera baseline and focal length [13,18,19].…”

“…In other words, there must be an assistive device that people with disabilities need to wear and be given a brief lesson on how to operate the tool, and then the viewing aid is ready to use [6,8,9]. In avoiding objects when navigating in a specific environment, the problem is how an aid device can accurately measure a distance [10,7,11]. Previous research was conducted to measure the distance using a stereo camera.…”

Autonomous and Sustainable Service Economies: Data-Driven Optimization of Design and Operations through Discovery of Multi-Perspective Parameters

“…Among the key benefits of IPS is their high level of accuracy. With an accuracy of up to several centimeters, IPS can help people navigate indoor spaces more efficiently and reduce the time spent searching for a specific location [220].…”

A Smart Clothing Approach for Augmenting Mobility of Visually Impaired People

“…Since then there has been aids for improving mobility of visually impaired persons. The recent works published in this domain [5,6,7,8] shows that the field is still an area of active research.…”

“…Carolyn [43] further employed LIDAR technology which could provide Spatial perception about object orientation and distance, however it was possible to measure the Object elevation due to 2D Lidar. Recently, the Lid Sonic system proposed by Sahar et al [44] employed Arduino edge computing device along with a Bluetooth-transmitting smartphone app. Arduino could collect data and detects obstacles using a combination of Lidar point cloud data and ultrasonic data, and provides users with sound feedback.…”