Single-Camera and Inter-Camera Vehicle Tracking and 3D Speed Estimation Based on Fusion of Visual and Semantic Features

Tang, Zheng; Wang, Gaoang; Xiao, H.; Zheng, Aotian; Hwang, Jenq‐Neng

doi:10.1109/cvprw.2018.00022

Cited by 132 publications

(87 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another state-of-the-art vehicle ReID method [43] is the winner of the vehicle ReID track in the AI City Challenge Workshop at CVPR 2018 [31], which is based on fusing visual and semantic features (FVS). This method extracts 1,024-dimension CNN features from a GoogLeNet [39] pre-trained on the CompCars benchmark [53].…”

Section: Image-based Reidmentioning

confidence: 99%

“…Tab 9 shows the results of various methods for spatiotemporal association, MTSC tracking, and image-based ReID on CityFlow. Note that PROVID [29] compares visual features first, then uses spatio-temporal information for re-ranking; whereas 2WGMMF [20] and FVS [43] first model the spatio-temporal transition based on online learning or manual measurements, and then perform imagebased ReID only on the confident pairs. Note also that, since only trajectories spanning multiple cameras are included in the evaluation, different from MTSC tracking, false positives are considered in the calculation of MTMC tracking accuracy.…”

Section: Mtmc Trackingmentioning

confidence: 99%

See 1 more Smart Citation

CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification

Tang

Naphade

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

321

179

View full text Add to dashboard Cite

Urban traffic optimization using traffic cameras as sensors is driving the need to advance state-of-the-art multitarget multi-camera (MTMC) tracking. This work introduces CityFlow, a city-scale traffic camera dataset consisting of more than 3 hours of synchronized HD videos from 40 cameras across 10 intersections, with the longest distance between two simultaneous cameras being 2.5 km. To the best of our knowledge, CityFlow is the largest-scale dataset in terms of spatial coverage and the number of cameras/videos in an urban environment. The dataset contains more than 200K annotated bounding boxes covering a wide range of scenes, viewing angles, vehicle models, and urban traffic flow conditions. Camera geometry and calibration information are provided to aid spatio-temporal analysis. In addition, a subset of the benchmark is made available for the task of image-based vehicle re-identification (ReID). We conducted an extensive experimental evaluation of baselines/state-of-the-art approaches in MTMC tracking, multi-target single-camera (MTSC) tracking, object detection, and image-based ReID on this dataset, analyzing the impact of different network architectures, loss functions, spatio-temporal models and their combinations on task effectiveness. An evaluation server is launched with the release of our benchmark at the 2019 AI City Challenge that allows researchers to compare the performance of their newest techniques. We expect this dataset to catalyze research in this field, propel the state-of-the-art forward, and lead to deployed traffic optimization(s) in the real world.

show abstract

Section: Image-based Reidmentioning

confidence: 99%

Section: Mtmc Trackingmentioning

confidence: 99%

CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification

Tang

Naphade

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

321

179

View full text Add to dashboard Cite

show abstract

“…The tracking method of [69] has originally been applied for human tracking [61], which was shown to suffer from severe identity-switching issues. There are also traditional approaches based on computer vision techniques that are still popular [22,72,73]. In such approaches, discriminative methods using hand-crafted features, such as scale-invariant feature transforms (SIFT), speeded up robust features (SURF), region-based features or edge-based features, are applied for re-identifying vehicles [74][75][76][77][78].…”

Section: Related Workmentioning

confidence: 99%

Orientation- and Scale-Invariant Multi-Vehicle Detection and Tracking from Unmanned Aerial Videos

2019

View full text Add to dashboard Cite

Along with the advancement of light-weight sensing and processing technologies, unmanned aerial vehicles (UAVs) have recently become popular platforms for intelligent traffic monitoring and control. UAV-mounted cameras can capture traffic-flow videos from various perspectives providing a comprehensive insight into road conditions. To analyze the traffic flow from remotely captured videos, a reliable and accurate vehicle detection-and-tracking approach is required. In this paper, we propose a deep-learning framework for vehicle detection and tracking from UAV videos for monitoring traffic flow in complex road structures. This approach is designed to be invariant to significant orientation and scale variations in the videos. The detection procedure is performed by fine-tuning a state-of-the-art object detector, You Only Look Once (YOLOv3), using several custom-labeled traffic datasets. Vehicle tracking is conducted following a tracking-by-detection paradigm, where deep appearance features are used for vehicle re-identification, and Kalman filtering is used for motion estimation. The proposed methodology is tested on a variety of real videos collected by UAVs under various conditions, e.g., in late afternoons with long vehicle shadows, in dawn with vehicles lights being on, over roundabouts and interchange roads where vehicle directions change considerably, and from various viewpoints where vehicles’ appearance undergo substantial perspective distortions. The proposed tracking-by-detection approach performs efficiently at 11 frames per second on color videos of 2720p resolution. Experiments demonstrated that high detection accuracy could be achieved with an average F1-score of 92.1%. Besides, the tracking technique performs accurately, with an average multiple-object tracking accuracy (MOTA) of 81.3%. The proposed approach also addressed the shortcomings of the state-of-the-art in multi-object tracking regarding frequent identity switching, resulting in a total of only one identity switch over every 305 tracked vehicles.

show abstract

“…Deep learning vehicle detection can be split into two different model strategies: 1) a single shot object detector (SSD, YOLO, YOLOv2, and YOLOv3) and 2) a region-based object detector (R-CNN, Fast R-CNN, and Faster R-CNN). Recent papers such as Tang et al [1] and Sang et al [2] demonstrate the success that YOLOv2 has had on object detection in the 2018 AI City Challenge. In this paper, a PyTorch version of Redmon's [3] YOLOv3 model is applied to vehicle images from the Nexar Challenge 2 dataset, NEXET [4].…”

Section: Introductionmentioning

confidence: 99%

PyTorch YOLOv3 Object Detection for Vehicle Identification

2020

IJRTE

View full text Add to dashboard Cite

Detecting real-world vehicle objects captured from car-mounted cameras requires manual labelling of video images. Previous vehicle object detection papers such as the winners of the 2018 AI City Challenge [1] used a training set of over 4,500 hand labelled images. In this paper, we attempt to automate this task by applying transfer learning to a YOLOv3 model trained on Imagenet and then re-trained on a set of stock car images and a small subset of hand labelled images taken from front-mounted dashboard camera videos. The mean Average Precision (mAP) of the validation set is used to determine the effectiveness of model vehicle classification. There is a significant variance issue between the validation and training set because the video images are taken in 1) various weather and lighting conditions and 2) the stock images have different image perspectives. The experimental results demonstrate that the YOLOv3 model can reach an overall 16.07% mAP after 60 epochs of training and can identify classes of vehicles that had few training examples in the dataset

show abstract

Single-Camera and Inter-Camera Vehicle Tracking and 3D Speed Estimation Based on Fusion of Visual and Semantic Features

Cited by 132 publications

References 14 publications

CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification

CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification

Orientation- and Scale-Invariant Multi-Vehicle Detection and Tracking from Unmanned Aerial Videos

PyTorch YOLOv3 Object Detection for Vehicle Identification

Contact Info

Product

Resources

About