POI: Multiple Object Tracking with High Performance Detection and Appearance Feature

Abstract

Abstract. Detection and learning based appearance feature play the central role in data association based multiple object tracking (MOT), but most recent MOT works usually ignore them and only focus on the hand-crafted feature and association algorithms. In this paper, we explore the high-performance detection and deep learning based appearance feature, and show that they lead to significantly better MOT results in both online and offline setting. We make our detection and appearance feature publicly available 1 . In the following part, we first summarize the detection and appearance feature, and then introduce our tracker named Person of Interest (POI), which has both online and offline version 2 . DetectionIn data association based MOT, the tracking performance is heavily affected by the detection results. We implement our detector based on Faster R-CNN [14]. In our implementation, the CNN model is fine-tuned from the VGG-16 on ImageNet. In considering the definition of MOTA in MOT16 [12], the sum of false negatives (FN) and false positives (FP) poses a large impact on the value of MOTA. In Table 1, we show that our detection optimization strategies lead to the significant decrease in the sum of FP and FN 3 .1 https://drive.google.com/open?id=0B5ACiy41McAHMjczS2p0dFg3emM 2 We use POI to denote our online tracker and KDNT to denote our offline tracker in submission. 3 We use detection score threshold 0.3 for Faster R-CNN and -1 for DPMv5 , labeling the ID of detection box with incremental integer, and evaluate FP and FN with MOT16 devkit.

Joint Monocular 3D Vehicle Detection and Tracking

“…w deep , w 2D , w 3D are the weights of appearance, 2D overlap, and 3D overlap. We utilize a mixture of those factors as the affinity across frames, similar to the design of POI [55]. Comparing to 2D tracking, 3D-oriented tracking is more robust to ego-motion, visual occlusion, overlapping, and re-appearances.…”

Evaluating Merging Strategies for Sampling-based Uncertainty Techniques in Object Detection

“…Spatial Affinity: To measure the spatial affinity between detections, the location and shape of bounding boxes can be compared. Intersection Over Union (IoU) is a well established method for measuring spatial affinity, and a Product Association Cost (PAC) [25] and Exponential Association Cost (EAC) (referred to as motion affinity and shape affinity in previous work [26]) have also been demonstrated in object tracking literature.…”

Using Panoramic Videos for Multi-Person Localization and Tracking In A 3D Panoramic Coordinate

“…Persons' appearance can be utilized as an important tracking cue. Although existing works exploit the entire body appearance [7,8], we suppose that only using the upper body appearance can alleviate occlusion problems in crowd scenes. Since 2D body poses are estimated in this work, the upper body image patches can be cropped accordingly.…”