FAMNet: Joint Learning of Feature, Affinity and Multi-Dimensional Assignment for Online Multiple Object Tracking

Abstract

Data association-based multiple object tracking (MOT) involves multiple separated modules processed or optimized differently, which results in complex method design and requires non-trivial tuning of parameters. In this paper, we present an end-to-end model, named FAMNet, where Feature extraction, Affinity estimation and Multi-dimensional assignment are refined in a single network. All layers in FAMNet are designed differentiable thus can be optimized jointly to learn the discriminative features and higher-order affinity model for robust MOT, which is supervised by the loss directly from the assignment ground truth. We also integrate single object tracking technique and a dedicated target management scheme into the FAMNet-based tracking system to further recover false negatives and inhibit noisy target candidates generated by the external detector. The proposed method is evaluated on a diverse set of benchmarks including MOT2015, MOT2017, KITTI-Car and UA-DETRAC, and achieves promising performance on all of them in comparison with state-of-the-arts.

Online Multiple Object Tracking Using Rule Distillated Siamese Random Forest

“…In the case of ruledistilled SiameseRF, a processing speed of approximately 1.3 Hz faster than the basic SiameseRF method was shown while maintaining the overall performance. For the MOT17 dataset, FAMNet [23] showed the fastest processing speed at 1.6 Hz among other three comparison methods. However, this method has a large speed difference of 10.8 Hz from the basic SiameseRF.…”

“…Tracking using a Siamese CNN for person reidentification in MOT has recently been studied [10][19]- [23]. A Siamese CNN applies the same network to the detection and tracker and calculates the similarity in the difference between output function values.…”

“…This method extends the Siamese network by applying a feature pyramid network to the plain Siamese architecture and by developing a new multi-level discriminative feature. Chu and Ling [23] proposed an endto-end Siamese based MOT model including feature extraction, affinity estimation, and a multi-dimensional assignment. To further improve the tracking robustness during tracking, it includes single object tracking and the prediction of a dedicated target management.…”

“…In the second experiment on the tracking performance, we compared the state-of-the-art methods with the proposed method using the MOT 17 dataset, as shown in Table V. To verify the effectiveness of the proposed tracking method, the same performance was measured using 10 state-of-the-art methods: (1) DMAN [6], (2) DEEP_TAMA [42] using deep temporal appearance matching, (3) STRN [46] applying a spatial-temporal relation network, (4) FAMNet [23] using a multi-object assignment, (5) Tracktor [21], (6) TrctrD17 [33], (7) YoonKJ17 [47] applying one-shot-learning, (8) the proposed SiameseRF without rule distillation, and (9) the proposed SiameseRF with 30% rule distillation. Unlike in Table IV, in Table V, public detection results are listed instead of private detection to evaluate a more accurate tracking performance.…”

Traffic flow estimation with data from a video surveillance camera

“…Most of the recent improvements in the MOT task involve fusing motion features with appearance one to better distinguish highly occluded objects and reidentify lost instances. The appearance clues usually come from convolutional neural networks [4,40]. However, Tang et al [35] showed that hand-crafted features, like the histogram of oriented gradients and colour histograms, might also be used if no training data is provided.…”