A Hierarchical Deep Architecture and Mini-batch Selection Method for Joint Traffic Sign and Light Detection

Abstract

Traffic light and sign detectors on autonomous cars are integral for road scene perception. The literature is abundant with deep learning networks that detect either lights or signs, not both, which makes them unsuitable for reallife deployment due to the limited graphics processing unit (GPU) memory and power available on embedded systems. The root cause of this issue is that no public dataset contains both traffic light and sign labels, which leads to difficulties in developing a joint detection framework. We present a deep hierarchical architecture in conjunction with a mini-batch proposal selection mechanism that allows a network to detect both traffic lights and signs from training on separate traffic light and sign datasets. Our method solves the overlapping issue where instances from one dataset are not labelled in the other dataset. We are the first to present a network that performs joint detection on traffic lights and signs. We measure our network on the Tsinghua-Tencent 100K benchmark for traffic sign detection and the Bosch Small Traffic Lights benchmark for traffic light detection and show it outperforms the existing Bosch Small Traffic light state-of-the-art method. We focus on autonomous car deployment and show our network is more suitable than others because of its low memory footprint and real-time image processing time. Qualitative results can be viewed at https://youtu.be/ YmogPzBXOw.

Traffic Light Recognition Using Deep Learning and Prior Maps for Autonomous Cars

“…A more detailed study on traffic light detection using YOLO can be found in [24]. Pon et al [16] detect simultaneously traffic lights and traffic signs with a modified version of the Faster R-CNN [12].…”

“…General purpose object detectors have been well explored for traffic related problems (such as detection of pedestrians, traffic signs, etc), and YOLO [11] and Faster R-CNN [12] are two of these state of the art detectors. Motivated by the advances in deep learning, some recent works [13]- [16] leveraged some state-of-the-art neural detectors to locate (and further recognize) traffic lights from 2D camera images. Other works, such as [17], combined prior maps with deep learning classification.…”

Recent advances in deep learning for object detection

“…Moreover, a video consists of hundreds of frames and thus con- There are also many other real-world applications based on object detection such as vehicle detection [227,228,229], traffic-sign detection [230,231] and skeleton detection [232,233].…”

A three-stage real-time detector for traffic signs in large panoramas

“…Its creators, Zhu et al [2], bring the branch of the network forward to the end of the 6th layer, as well as terminating the network in three streams, which enables their architecture to simultaneously detect and classify traffic signs, and outperform Fast R-CNN. Lu [22], a hierarchical network is employed to identify the class in a coarse-to-fine way, and they successfully settled the issue of combination and overlapping of datasets, which allows their architecture to detect traffic signs and lights jointly. At the expense of performance, the model is able to perform inference at more than 60 fps, much faster than all previous detectors evaluated on TT100K.…”