A three-stage real-time detector for traffic signs in large panoramas

Abstract

Traffic sign detection is one of the key components in autonomous driving. Advanced autonomous vehicles armed with high quality sensors capture high definition images for further analysis.Detecting traffic signs, moving vehicles, and lanes is important for localization and decision making. Traffic signs, especially those that are far from the camera, are small, and so are challenging to traditional object detection methods. In this work, in order to reduce computational cost and improve detection performance, we split the large input images into small blocks and then recognize traffic signs in the blocks using another detection module. Therefore, this paper proposes a three-stage traffic sign detector, which connects a BlockNet with an RPN-RCNN detection network. BlockNet, which is composed of a set of CNN layers, is capable of performing block-level foreground detection, making inferences in less than 1 ms. Then, the RPN-RCNN two-stage detector is used to identify traffic sign objects in each block; it is trained on a derived dataset named TT100KPatch. Experiments show that our framework can achieve both state-of-the-art accuracy and recall; its fastest detection speed is 102 fps.

Real-time facial pose estimation and tracking by coarse-to-fine iterative optimization

“…This approach combined human skeleton tracking and video understanding, and achieved great performance. Song et al [20] designed a threestage real-time detector that responds well to small objects, such as traffic signs, especially those that are far from the camera, which is therefore challenging to traditional object detection methods. Our method benefits from the above advantages.…”

Dynamic Refinement Network for Oriented and Densely Packed Object Detection

“…Most object detection methods [35,27,32,36,34,18,44,37] focus on axis-aligned or upright objects and may encounter problems when the targets are of arbitrary orientations or present dense distribution [9]. For oriented object detection, some methods [8,10,25,29,28] adopt the R-CNN [35] framework and use numerous anchors with different angles, scales, and aspect ratios, at the expense of considerably increasing computation complexity.…”

Equirectangular Image Data Detection, Segmentation and Classification of Varying Sized Traffic Signs: A Comparison of Deep Learning Methods

“…Current image datasets such as The German Traffic Sign Detection Benchmark [ 10 ], German Traffic Sign Recognition Benchmark [ 11 ], TRMSD [ 12 ], COCO [ 13 ], ImageNet [ 14 ], Cityscapes [ 15 ], PASCAL VOC, [ 16 ] PASCAL-Context [ 17 ], and KITTI [ 18 ] do not include omnidirectional images. As such, methods that are modeled on these datasets often do not perform well when applied to omnidirectional images from the real world [ 19 ]. This is because these methods do not learn from datasets that include images that consider temporal environmental conditions, the traffic signs are often smaller in images captured in the real world but mostly fill the size of the image in these datasets, and omnidirectional images introduce further distortion to traffic signs based on their locality to the camera system.…”

“…It is typical to split deep learning object detection methods into proposal-free and proposal-based methods [ 19 ]. We do not use proposal-free methods in our paper so do not discuss these further.…”

“…Proposal-based methods are not very effective for smaller objects due to reduced pixel information, resulting in low-quality traffic sign candidate images for external classifiers. To overcome this, a three-stage traffic sign detector was proposed [ 19 ] connecting BlockNet [ 26 ] with the RPN-RCNN detection network [ 27 ]. BlockNet was employed to divide each input image into 256 × 256 partitions that overlap each other before passing into six convolutional layers for binomial classification of foreground and background using a softmax layer.…”