LYTNet: A Convolutional Neural Network for Real-Time Pedestrian Traffic Lights and Zebra Crossing Recognition for the Visually Impaired

Abstract

Currently, the visually impaired rely on either a sighted human, guide dog, or white cane to safely navigate. However, the training of guide dogs is extremely expensive, and canes cannot provide essential information regarding the color of traffic lights and direction of crosswalks. In this paper, we propose a deep learning based solution that provides information regarding the traffic light mode and the position of the zebra crossing. Previous solutions that utilize machine learning only provide one piece of information and are mostly binary: only detecting red or green lights. The proposed convolutional neural network, LYTNet, is designed for comprehensiveness, accuracy, and computational efficiency. LYTNet delivers both of the two most important pieces of information for the visually impaired to cross the road. We provide five classes of pedestrian traffic lights rather than the commonly seen three or four, and a direction vector representing the midline of the zebra crossing that is converted from the 2D image plane to real-world positions. We created our own dataset of pedestrian traffic lights containing over 5000 photos taken at hundreds of intersections in Shanghai. The experiments carried out achieve a classification accuracy of 94%, average angle error of 6.35 • , with a frame rate of 20 frames per second when testing the network on an iPhone 7 with additional post-processing steps.

Zebra Crossing Detection Using Concatenation of SIFT-Gabor Features

“…The design of a low-power, low-latency electronic mobility assistance for blind persons revealed that decision trees, random forests, and KNNs may all be used to recognise objects [10]. The pedestrian walk detection system is adopted by HOG and LBPH methods worked together with the SVM algorithm [11] where which uses segmentation according to the color of the pixel, classifies shape using linear SVM performs form content recognition using Gaussian-kernel SVMs [12] and to model the video they use GMM [13] which depends on YOLOv3 real-time data and applied kinematic based filter [14] for vehicle detection and to obtain high accuracy for vehicle speed SVM and HOG [15] methods are used [16]. Many application based model have been developed one of those is, An Electronic Travel Aid for Navigation of Visually Impaired Persons [17], a means through which a blind person can autonomously navigate a new environment.…”

A regression approach to zebra crossing detection based on convolutional neural networks

“…In Ghilardi et al [13], the support vector machine (SVM) algorithm is adopted to perform crosswalk classification on the low‐resolution satellite images. In Yu et al [14], the LYTNet is proposed to provide the mode of the traffic light and the start‐end points of the zebra crossing. Notwithstanding the good performances of the existing works in localizing the zebra crossing, the direct estimation of the zebra crossing still needs further studies.…”