Smart Navigation Detection using Deep-learning for Visually Impaired Person

Kumar, Nitin; Jain, Anuj

doi:10.1109/icepes52894.2021.9699479

Cited by 5 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We associate the trained CNN model (Kumar & Jain, 2021) with the live camera feed once the model has been trained. The bounding box for the detected area is drawn if the object is detected.…”

Section: Proposed Methodologymentioning

confidence: 99%

A Deep Learning Based Model to Assist Blind People in Their Navigation

Kumar

Jain

2022

JITE:IIP

Self Cite

View full text Add to dashboard Cite

Aim/Purpose: This paper proposes a new approach to developing a deep learning-based prototyping wearable model which can assist blind and visually disabled people to recognize their environments and navigate through them. As a result, visually impaired people will be able to manage day-to-day activities and navigate through the world around them more easily. Background: In recent decades, the development of navigational devices has posed challenges for researchers to design smart guidance systems for visually impaired and blind individuals in navigating through known or unknown environments. Efforts need to be made to analyze the existing research from a historical perspective. Early studies of electronic travel aids should be integrated with the use of assistive technology-based artificial vision models for visually impaired persons. Methodology: This paper is an advancement of our previous research work, where we performed a sensor-based navigation system. In this research, the navigation of the visually disabled person is carried out with a vision-based 3D-designed wearable model and a vision-based smart stick. The wearable model used a neural network-based You Only Look Once (YOLO) algorithm to detect the course of the navigational path which is augmented by a GPS-based smart Stick. Over 100 images of each of the three classes, namely straight path, left path and right path, are being trained using supervised learning. The model accurately predicts a straight path with 79% mean average precision (mAP), the right path with 83% mAP, and the left path with 85% mAP. The average accuracy of the wearable model is 82.33% and that of the smart stick is 96.14% which combined gives an overall accuracy of 89.24%. Contribution: This research contributes to the design of a low-cost navigational standalone system that will be handy to use and help people to navigate safely in real-time scenarios. The challenging self-built dataset of various paths is generated and transfer learning is performed on the YOLO-v5 model after augmentation and manual annotation. To analyze and evaluate the model, various metrics, such as model losses, recall value, precision, and maP, are used. Findings: These were the main findings of the study: • To detect objects, the deep learning model uses a higher version of YOLO, i.e., a YOLOv5 detector, that may help those with visual im-pairments to improve their quality of navigational mobilities in known or unknown environments. • The developed standalone model has an option to be integrated into any other assistive applications like Electronic Travel Aids (ETAs) • It is the single neural network technology that allows the model to achieve high levels of detection accuracy of around 0.823 mAP with a custom dataset as compared to 0.895 with the COCO dataset. Due to its lightning-speed of 45 FPS object detection technology, it has become popular. Recommendations for Practitioners: Practitioners can help the model’s efficiency by increasing the sample size and classes used in training the model. Recommendation for Researchers: To detect objects in an image or live cam, there are various algorithms, e.g., R-CNN, Retina Net, Single Shot Detector (SSD), YOLO. Researchers can choose to use the YOLO version owing to its superior performance. Moreover, one of the YOLO versions, YOLOv5, outperforms its other versions such as YOLOv3 and YOLOv4 in terms of speed and accuracy. Impact on Society: We discuss new low-cost technologies that enable visually impaired people to navigate effectively in indoor environments. Future Research: The future of deep learning could incorporate recurrent neural networks on a larger set of data with special AI-based processors to avoid latency.

show abstract

Section: Proposed Methodologymentioning

confidence: 99%

A Deep Learning Based Model to Assist Blind People in Their Navigation

Kumar

Jain

2022

JITE:IIP

Self Cite

View full text Add to dashboard Cite

show abstract

“…N. Kumar et al proposed a wearable mask and a stick, which can achieve path edge detection applying the YOLOv3 algorithm, whereas the dataset contains only 25 categories. It is worth mentioning that this work uses bone conduction headphones to provide information feedback to the user [10]. The wearable assistive system proposed by W. -J. Chang et al consists of a pair of sunglasses, a waistmounted device, and a walking stick.…”

Section: Ralated Researchmentioning

confidence: 99%

IBGS: A Wearable Smart System to Assist Visually Challenged

Xia

Liu

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Traditional blind guide devices are large and high cost. In this paper, a kind of intelligent blind guide system (IBGS) is introduced. GD32 is used as the main control chip, it cooperates with various functional modules to realize traffic light recognition, obstacle avoidance, payment, and navigation functions on the basis of speech recognition. At the same time, IBGS uses WIFI instead of Bluetooth to get rid of the dependence on smart phones. In addition, a cloud database is built, and the Internet of Things technology is used to realize the information interconnection between the IBGS, the database, and the guardian's mobile terminal. In order to better realize the function of speech recognition, this paper proposes a Conv-Transformer Transducer (ConvT-T) speech recognition framework based on Weak-Attention Suppression (WAS), which improves the efficiency of multi-head attention through WAS. The proposed method achieves word error rate (WER) of 3.2% in test-clean and 7.9% in test-other on the LibriSpeech ASR corpus with only 73M parameters, which reduces the WER by 0.3% and 0.4% compared to ConvT-T respectively, indicating that WAS can effectively play the role of suppressing non-critical attention on the ConvT-T framework. At the same time, the IBGS has been tested in a comprehensive application scenario. In outdoor traffic light recognition, speech recognition, and obstacle avoidance tests, the accuracy rates of 92.33%, 90.33%, and 96.67% were achieved respectively. The results show that the IBGS can effectively deal with various challenges encountered by the blind in their daily outdoor walking.

show abstract