Multi-person Tracking by Multicut and Deep Matching

Abstract

Abstract. In [1], we proposed a graph-based formulation that links and clusters person hypotheses over time by solving a minimum cost subgraph multicut problem. In this paper, we modify and extend [1] in three ways: 1) We introduce a novel local pairwise feature based on local appearance matching that is robust to partial occlusion and camera motion. 2) We perform extensive experiments to compare different pairwise potentials and to analyze the robustness of the tracking formulation. 3) We consider a plain multicut problem and remove outlying clusters from its solution. This allows us to employ an efficient primal feasible optimization algorithm that is not applicable to the subgraph multicut problem of [1]. Unlike the branch-and-cut algorithm used there, this efficient algorithm used here is applicable to long videos and many detections. Together with the novel feature, it eliminates the need for the intermediate tracklet representation of [1]. We demonstrate the effectiveness of our overall approach on the MOT16 benchmark [2], achieving state-of-art performance.

PoseTrack: Joint Multi-person Pose Estimation and Tracking

“…To that end, we build on the idea recently used in multi-person tracking [38] and compute dense correspondences between two frames using DeepMatching [41].…”

“…We also append the feature vector with non-linear terms as done in [38]. The mapping from the feature vector to the probability p …”

“…We extend it to videos as follows. We first generate person bounding boxes for all frames in the video using a state-ofthe-art person detector (Faster R-CNN [34]), and perform person tracking using a state-of-the-art person tracker [38] and train it on the training set of the Multi-Person PoseTrack Dataset. We also discard all tracks that are shorter than 7 frames.…”

“…Interestingly, the person boundingbox tracking based baselines achieve a lower number of ID switches. We believe that this is primarily due to the powerful multi-target tracking approach [38], which can handle person identities more robustly. Figure 4: Left Impact of the the temporal edge density.…”

Multiple Object Tracking with Motion and Appearance Cues

“…Data association is a key step in tracking-by-detection based MOT methods. Many offline MOT methods [24,25,26] treat data associations as graph-based optimization problems. Hungarian algorithm [27] is another commonly used data association optimization method.…”