Efficient Multi-Object Detection and Smart Navigation Using Artificial Intelligence for Visually Impaired People

Abstract

Visually impaired people face numerous difficulties in their daily life, and technological interventions may assist them to meet these challenges. This paper proposes an artificial intelligence-based fully automatic assistive technology to recognize different objects, and auditory inputs are provided to the user in real time, which gives better understanding to the visually impaired person about their surroundings. A deep-learning model is trained with multiple images of objects that are highly relevant to the visually impaired person. Training images are augmented and manually annotated to bring more robustness to the trained model. In addition to computer vision-based techniques for object recognition, a distance-measuring sensor is integrated to make the device more comprehensive by recognizing obstacles while navigating from one place to another. The auditory information that is conveyed to the user after scene segmentation and obstacle identification is optimized to obtain more information in less time for faster processing of video frames. The average accuracy of this proposed method is 95.19% and 99.69% for object detection and recognition, respectively. The time complexity is low, allowing a user to perceive the surrounding scene in real time.

Inter-Urban Analysis of Pedestrian and Drivers through a Vehicular Network Based on Hybrid Communications Embedded in a Portable Car System and Advanced Image Processing Technologies

“…This mini-network contains only seven convolutional layers and approximately six grouping layers. We can say that it is about five times slower in the sampling required to obtain ideal detection processes [67,68]. The structure of this network outlines through its annexes a mini-network that can later increase speed but loses accuracy.…”

A Systematic Review of Urban Navigation Systems for Visually Impaired People

“…Duh et al [ 39 ] and Yang et al [ 47 ] used semantic segmentation to recognize obstacles. While Lin et al [ 61 ] switched between Faster R-CNN and YOLO on different modes, Joshi et al [ 68 ] used YOLO-v3. Chun et al [ 26 ] used laser (LiDAR) sensor measures to define the types of hazards (staircase, ramp, drainage, pothole, and step).…”

“…The number of obstacles defines the number of covered objects in each dataset whether they were applied for a BVIP use case or not. As shown in the table, there is no dataset that defines all needed obstacles from BVIP’s perspectives [ 44 , 60 , 68 ]. Although Lin et al [ 44 ] built a dataset with 6000 obstacles for BVIP’s usage, this dataset contains only low-lying obstacles.…”

Computer Vision Based Automatic Recognition of Pointer Instruments: Data Set Optimization and Reading

“…The position coordinates and category probability of the target can be generated through the CNN to obtain the position coordinates and corresponding confidence of the target directly from the picture. Typical algorithms of the one-stage model include You Only Look Once (YOLO) [16] and SSD [17].…”

“…The YOLO series pursues detection speed at the expense of accuracy. The SSD algorithm strikes a balance between speed and accuracy, gaining a greater increase in speed at the expense of less decrease in accuracy [16,17]. However, in principle, the meter reading system in the substation does not allow false negatives, so this paper selects the Faster R-CNN model with higher detection accuracy and faster rate…”