TransTrack: Multiple Object Tracking with Transformer

Abstract

Multiple-object tracking(MOT) is mostly dominated by complex and multi-step tracking-by-detection algorithm, which performs object detection, feature extraction and temporal association, separately. Query-key mechanism in single-object tracking(SOT), which tracks the object of the current frame by object feature of the previous frame, has great potential to set up a simple joint-detectionand-tracking MOT paradigm. Nonetheless, the query-key method is seldom studied due to its inability to detect newcoming objects.In this work, we propose TransTrack, a baseline for MOT with Transformer. It takes advantage of query-key mechanism and introduces a set of learned object queries into the pipeline to enable detecting new-coming objects. TransTrack has three main advantages: (1) It is an online joint-detection-and-tracking pipeline based on querykey mechanism. Complex and multi-step components in the previous methods are simplified. (2) It is a brand new architecture based on Transformer. The learned object query detects objects in the current frame. The object feature query from the previous frame associates those current objects with the previous ones. (3) For the first time, we demonstrate a much simple and effective method based on query-key mechanism and Transformer architecture could achieve competitive 65.8% MOTA on the MOT17 challenge dataset. We hope TransTrack can provide a new perspective for multiple-object tracking. The code is available at: https://github.com/PeizeSun/TransTrack.

MAXIM: Multi-Axis MLP for Image Processing

“…The elegance of ViT [23] has also motivated similar model designs with simpler global operators such as MLP-Mixer [85], gMLP [53], GFNet [74], and FNet [43], to name a few. Despite successful applications to many high-level tasks [4,23,56,83,87,100], the efficacy of these global models on low-level enhancement and restoration problems has not been studied extensively. The pioneering works on Transformers for lowlevel vision [9,14] directly applied full self-attention, which only accepts relatively small patches of fixed sizes (e.g., 48×48).…”

CMC2R: Cross‐modal collaborative contextual representation for RGBT tracking

“…Track-RCNN [22] and FairMOT [23] further add a Re-ID branch on top of object detector in a joint training framework, incorporating object detection and Re-ID feature learning. Based on DETR, TransTrack [9] and TrackFormer [24] develop the transformer-based frameworks for MOT.…”

“…Based on transformer-based methods [9], and taking the heterogenicity of different modalities, our CMC2R is a fully endto-end framework, which fuses the information collaboratively using the two-stream structure and the transformer structure, and the detection and tracking is trained jointly. Secondly, The NMS is not needed for tracks association, and the temporal passing module combined with multi-frame tracking feature is proposed to model the temporal relation.…”

“…It proposes the concept of "object query", which is an explicit and decoupled representation of object. Motivated by DETR's great success, TransTrack [9] extends the "object query" concept to model object tracking, called track query. Each track query is responsible to predict an entire track of an object.…”

“…Each track query is responsible to predict an entire track of an object. Motivated by [9] and the aforementioned issues, this paper proposes an RGB-T object tracking method based on query-key mechanism and JDT paradigm.…”