BoxCars: Improving Fine-Grained Recognition of Vehicles Using 3-D Bounding Boxes in Traffic Surveillance

Abstract

In this paper, we focus on fine-grained recognition of vehicles mainly in traffic surveillance applications. We propose an approach that is orthogonal to recent advancements in finegrained recognition (automatic part discovery, bilinear pooling). Also, in contrast to other methods focused on fine-grained recognition of vehicles, we do not limit ourselves to a frontal/rear viewpoint, but allow the vehicles to be seen from any viewpoint. Our approach is based on 3D bounding boxes built around the vehicles. The bounding box can be automatically constructed from traffic surveillance data. For scenarios where it is not possible to use precise construction, we propose a method for an estimation of the 3D bounding box. The 3D bounding box is used to normalize the image viewpoint by "unpacking" the image into a plane. We also propose to randomly alter the color of the image and add a rectangle with random noise to a random position in the image during the training of Convolutional Neural Networks. We have collected a large fine-grained vehicle dataset BoxCars116k, with 116k images of vehicles from various viewpoints taken by numerous surveillance cameras. We performed a number of experiments which show that our proposed method significantly improves CNN classification accuracy (the accuracy is increased by up to 12 percentage points and the error is reduced by up to 50 % compared to CNNs without the proposed modifications). We also show that our method outperforms stateof-the-art methods for fine-grained recognition.

Vehicle Attribute Recognition by Appearance: Computer Vision Methods for Vehicle Type, Make and Model Classification

“…Sochor et al are noted for collecting the BoxCars [64] and BoxCars116k [65] datasets. In [64], the recognition performance is boosted by inserting additional supplementary information to the neural network, more specifically, 3D vehicle bounding box, rasterized low-resolution shape, and 3D vehicle orientation.…”

Vision-Based Potential Pedestrian Risk Analysis on Unsignalized Crosswalk Using Data Mining Techniques

“…In one case, Sochor et al proposed a model constructing 3-D bounding boxes around vehicles through the use of convolutional neural networks (CNNs) from only a single camera viewpoint. This makes it possible to project the coordinates of the car from an oblique viewpoint to dimensionally-accurate space [16]. Likewise, Hussein et al tracked pedestrians from two hours of video data from a major signalized In the second module, we extract various features frame-by-frame, such as vehicle velocity, vehicle acceleration, pedestrian velocity, the distance between vehicle and pedestrian, and the distance between vehicle and crosswalk.…”

Learning feature aggregation in temporal domain for re-identification

“…There are datasets of vehicles (Krause et al, 2013;Yang et al, 2015;Sochor et al, 2017), which are created for finegrained recognition with annotations on several attributes such as type, make and color. However, the identities of the vehicles in the datasets are not known; thus, the datasets are not directly applicable for vehicle re-identification, especially for evaluation.…”

“…For feature extraction from images we use Inception-ResNet-v2 (Szegedy et al, 2017) with images resized to 331 × 331 yielding feature vectors with length 1536 for each input image. Sochor et al (2016Sochor et al ( , 2017 showed that unpacking the input vehicle by 3D bounding box and alternating the input image colors is beneficial for fine-grained recognition of vehicles; we use these modifications for re-identification of vehicles as well.…”