Traffic Light Recognition Using Deep Learning and Prior Maps for Autonomous Cars

Abstract

Autonomous terrestrial vehicles must be capable of perceiving traffic lights and recognizing their current states to share the streets with human drivers. Most of the time, human drivers can easily identify the relevant traffic lights. To deal with this issue, a common solution for autonomous cars is to integrate recognition with prior maps. However, additional solution is required for the detection and recognition of the traffic light. Deep learning techniques have showed great performance and power of generalization including traffic related problems. Motivated by the advances in deep learning, some recent works leveraged some state-of-the-art deep detectors to locate (and further recognize) traffic lights from 2D camera images. However, none of them combine the power of the deep learning-based detectors with prior maps to recognize the state of the relevant traffic lights. Based on that, this work proposes to integrate the power of deep learning-based detection with the prior maps used by our car platform IARA (acronym for Intelligent Autonomous Robotic Automobile) to recognize the relevant traffic lights of predefined routes. The process is divided in two phases: an offline phase for map construction and traffic lights annotation; and an online phase for traffic light recognition and identification of the relevant ones. The proposed system was evaluated on five test cases (routes) in the city of Vitória, each case being composed of a video sequence and a prior map with the relevant traffic lights for the route. Results showed that the proposed technique is able to correctly identify the relevant traffic light along the trajectory.

An improved traffic lights recognition algorithm for autonomous driving in complex scenarios

“…Based on the GPS signal-assisted acquisition of the ROI and using the convolutional neural network for feature extraction and classification, John et al 9 proposed two image recognition schemes suitable for normal light and low light conditions, respectively. Possatti et al 42 Combined the newly proposed deep learning algorithm and prior knowledge assistance and used a prior map to select relevant traffic lights of vehicle driving behavior from the recognition results of YOLOv3.…”

Deep traffic light detection by overlaying synthetic context on arbitrary natural images

“…More recently, deep neural networks (DNNs) have been shown quite effective in various autonomous driving tasks, including traffic light detection. Some works show that generic object detectors, such as YOLO [20] and Faster R-CNN [19] are effective for traffic light detection [27]. Behrendt et al [28] modified the training procedure of YOLO [20] to better handle issues more specific to traffic light detection, such as the small size of the objects of interest.…”

Real-Time Traffic Light Detection with Frequency Patterns Using a High-Speed Camera

“…Some methods detect traffic lights and classify which lane the traffic light corresponds to using CNN [ 10 , 11 ]. In [ 22 ], traffic light detection was based on a deep neural net and a prior map. They used a prior map to select traffic lights corresponding to the vehicle’s current lane among the lights detected by the network.…”

“…In some cases, high recognition accuracy is achieved by fusing HOG features and features extracted by CNN [ 9 ]. Network models such as RetinaNet [ 26 ], and YOLO [ 8 , 22 ] have also been studied. Meanwhile, a method for designing an original background suppression filter and learning filter coefficients using numerous traffic light images without a neural network was proposed [ 27 ].…”