Online Multi-Object Tracking with Dual Matching Attention Networks

Abstract

In this paper, we propose an online Multi-Object Tracking (MOT) approach which integrates the merits of single object tracking and data association methods in a unified framework to handle noisy detections and frequent interactions between targets. Specifically, for applying single object tracking in MOT, we introduce a cost-sensitive tracking loss based on the state-of-the-art visual tracker, which encourages the model to focus on hard negative distractors during online learning. For data association, we propose Dual Matching Attention Networks (DMAN) with both spatial and temporal attention mechanisms. The spatial attention module generates dual attention maps which enable the network to focus on the matching patterns of the input image pair, while the temporal attention module adaptively allocates different levels of attention to different samples in the tracklet to suppress noisy observations. Experimental results on the MOT benchmark datasets show that the proposed algorithm performs favorably against both online and offline trackers in terms of identity-preserving metrics.

Deep learning in video multi-object tracking: A survey

“…Those features were further processed by a bidirectional LSTM. In [115] a similar approach was followed, with a so-called Spatial Attention Network (SAN). The SAN was a Siamese CNN, which used a pretrained ResNet-50 as base model.…”

Aerial Multi-Object Tracking by Detection Using Deep Association Networks

“…In [25], in order to efficiently learn the long-term appearance models via a recurrent network, Bilinear LSTM based technique is proposed. In [55], authors utilize the advantages of single object tracking and data association methods to detect and track objects in noisy environments. In [18], authors postulate the tracking problem as a weighted graph labeling problem.…”

State-Aware Re-Identification Feature for Multi-Target Multi-Camera Tracking

“…In [9], Chu et al introduce the SOT in MOT framework, and spatial-temporal attention mechanism (STAM) is adopted to handle the drift problems caused by SOT. In [63], Zhu et al propose an extended Efficient Convolution Operators (ECO) [10] with cost-sensitive tracking loss and introduce Dual Matching Attention Networks (DMAN) with both spatial and temporal attention mechanisms for data association.…”

“…In the context of appearance feature, many works [38,8,13,63,3,57] recently adopt deep learning to represent appearance of the target. In [13], Feng et al design a qualityaware mechanism to select the K images from the historical samples of the target, and ResNet-18 [17] is adopted to measure the quality of the detection.…”

“…In this paper, we concentrate on the first two components. Appearance feature is significant to maintain the identity of the tracked target, and many works [9,63,13] have exploited the reliable appearance model. In specific, color histogram [33,20] and HOG [27,7] are well studied and utilized in previous works.…”