CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification

Abstract

Urban traffic optimization using traffic cameras as sensors is driving the need to advance state-of-the-art multitarget multi-camera (MTMC) tracking. This work introduces CityFlow, a city-scale traffic camera dataset consisting of more than 3 hours of synchronized HD videos from 40 cameras across 10 intersections, with the longest distance between two simultaneous cameras being 2.5 km. To the best of our knowledge, CityFlow is the largest-scale dataset in terms of spatial coverage and the number of cameras/videos in an urban environment. The dataset contains more than 200K annotated bounding boxes covering a wide range of scenes, viewing angles, vehicle models, and urban traffic flow conditions. Camera geometry and calibration information are provided to aid spatio-temporal analysis. In addition, a subset of the benchmark is made available for the task of image-based vehicle re-identification (ReID). We conducted an extensive experimental evaluation of baselines/state-of-the-art approaches in MTMC tracking, multi-target single-camera (MTSC) tracking, object detection, and image-based ReID on this dataset, analyzing the impact of different network architectures, loss functions, spatio-temporal models and their combinations on task effectiveness. An evaluation server is launched with the release of our benchmark at the 2019 AI City Challenge that allows researchers to compare the performance of their newest techniques. We expect this dataset to catalyze research in this field, propel the state-of-the-art forward, and lead to deployed traffic optimization(s) in the real world.

PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data

“…3. Our proposed method achieves significant improvement over the state-of-the-art on two mainstream benchmarks: VeRi [14] and CityFlow-ReID [30]. Additional experiments validate that our unique architecture exploiting explicit pose information, along with our use of randomized synthetic data for training, are key to the method's success.…”

“…Hence, we also generate a highly randomized synthetic dataset, in which a large variety of viewing angles and random noise such as strong shadow, occlusion, and cropped images are simulated. Finally, extensive experiments are conducted on VeRi [14] and CityFlow-ReID [30] to evaluate PAMTRI against state-of-the-art in vehicle ReID. Our proposed framework achieves the top performance in both benchmarks, and an ablation study shows that each proposed component helps enhance robustness.…”

“…The final generated dataset consists of 41,000 unique images with 402 identities, 1 including the following annotations: keypoints, orientation, and vehicle attributes (color and type). When generating the dataset, background images were sampled from CityFlow [30], and we also randomized the vehicle position and intensity of light. Additionally, during training we perform randomized post-processing such as scaling, cropping, horizontal flipping, and adding occlusion.…”

“…Recent methods to vehicle ReID exploit feature learning [37,45,46] and/or distance metric learning [3,10,13] to train deep neural networks (DNNs) to distinguish between vehicle pairs, but the current state-of-the-art performance is still far from its counterpart in person ReID [43]. Moreover, it has been shown [30] that directly using state-ofthe-art person ReID methods for vehicles does not close this gap, indicating fundamental differences between the two tasks. We believe the key to vehicle ReID is to exploit viewpoint-invariant information such as color, type, and deformable shape models encoding pose.…”

A Strong and Efficient Baseline for Vehicle Re-Identification Using Deep Triplet Embedding

“…There are about 71k and 36k identities in VD1 and VD2, respectively. CityFlow: Presented at CVPR 2019 [50], this large scale dataset is the biggest city-wide dataset collected in USA. The dataset comprises annotations for the tasks of re-identification, multi-camera-CNN Embedding multi-target vehicle tracking, containing more than 200K annotated bounding boxes covering a wide range of scenes, viewing angles, vehicle models, and urban traffic flow conditions.…”