Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking

Cao, Jinkun; Weng, Xinshuo; Khirodkar, Rawal; Pang, Jiangmiao

doi:10.48550/arxiv.2203.14360

Cited by 33 publications

(54 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, several recent studies [63,64,94,7] have abandoned appearance information, and relied only on highperformance detectors and motion information, which achieve high running speed and state-of-the-art performance on MOTChallenge benchmarks [48,14]. However, abandoning appearance features would lead to poor robustness in more complex scenes.…”

Section: Related Work a Separate And Joint Trackersmentioning

confidence: 99%

StrongSORT: Make DeepSORT Great Again

Zhao

Song

et al. 2023

IEEE Trans. Multimedia

281

View full text Add to dashboard Cite

Multi-Object Tracking (MOT) has gained lots attention from researchers and achieved remarkable progress in recent years. However, recent studies on MOT tend to use different basic models (e.g, detector and embedding model), training data and training/inference tricks, which makes it difficult to construct a fair comparison between their progress. In this paper, we revisit the classic tracker DeepSORT, and upgrades it from aspects of detection, embedding, and association. The proposed tracker, named StrongSORT, achieves great improvements over DeepSORT, and can serve as a strong and fair baseline for other methods. We also present two lightweight and plugand-play algorithms to solve two "missing" problems in MOT. Firstly, to solve the missing association problem, some works associate short tracklets into complete trajectories with computationally expensive models. Instead, we propose the appearancefree link model (AFLink) to perform global association without appearance information, which achieves a better trade-off between speed and accuracy. Secondly, for the missing detection problem, we propose Gaussian-smoothed interpolation (GSI), which improves the linear interpolation algorithm with Gaussian process regression. Moreover, AFLink and GSI can be plugged into various trackers with a negligible extra computational cost (1.7 ms and 7.1 ms per image, respectively, on MOT17). By integrating StrongSORT with the two algorithms, the final tracker StrongSORT++ achieves SOTA results on multiple benchmarks, i.e., MOT17, MOT20, DanceTrack and KITTI. Codes are available at https://github.com/dyhBUPT/StrongSORT and https://github.com/open-mmlab/mmtracking.

show abstract

Section: Related Work a Separate And Joint Trackersmentioning

confidence: 99%

StrongSORT: Make DeepSORT Great Again

Zhao

Song

et al. 2023

IEEE Trans. Multimedia

281

View full text Add to dashboard Cite

show abstract

“…Both only have training and testing sets, while validation sets are not available. Therefore, in the ablation experiments, we follow the experimental setup in most of the literature [1], [4], [5], [13]. We divide each video in the MOT17 training set into two equal parts.…”

Section: A Datasets and Metricsmentioning

confidence: 99%

“…In past studies on MOT tasks, the motion information was encoded either by conventional filtering or data-driven methods. Compared with conventional filtering [1], [4], [13]- [18], data-driven methods [19]- [23] have attracted less attention since most studies typically use motion models as an auxiliary clue. However, data-driven motion models have inherent advantages in handling nonlinear and sophisticated motion patterns which can not be expressed by filtering-based methods.…”

Section: Introductionmentioning

confidence: 99%

STDFormer: Spatial-Temporal Motion Transformer for Multiple Object Tracking

Zhu

Wang

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Mainstream multi-object tracking methods exploit appearance information and/or motion information to achieve interframe association. However, dealing with similar appearance and occlusion is a challenge for appearance information, while motion information is limited by linear assumptions and is prone to failure in nonlinear motion patterns. In this work, we disregard appearance clues and propose a pure motion tracker to address the above issues. It dexterously utilizes Transformer to estimate complex motion and achieves high-performance tracking with low computing resources. Furthermore, contrastive learning is introduced to optimize feature representation for robust association. Specifically, we first exploit the long-range modeling capability of Transformer to mine intention information in temporal motion and decision information in spatial interaction and introduce prior detection to constrain the range of motion estimation. Then, we introduce contrastive learning as an auxiliary task to extract reliable motion features to compute affinity and introduce bidirectional matching to improve the affinity computation distribution. In addition, given that both tasks are dedicated to narrowing the embedding distance between the motion features of the tracked object and the detection features, we design a joint-motion-and-association framework to unify the above two tasks in one framework for optimization. The experimental results achieved with three benchmark datasets, MOT17, MOT20 and DanceTrack, verify the effectiveness of our proposed method. Compared with state-of-the-art methods, the proposed STDFormer sets a new state-of-the-art on DanceTrack and achieves competitive performance on MOT17 and MOT20. This demonstrates the advantage of our method in handling associations under similar appearance, occlusion or nonlinear motion. At the same time, the significant advantages of the proposed method over Transformer-based and contrastive learning-based methods suggest a new direction for the application of Transformer and contrastive learning in MOT. In addition, to verify the generalization of STDFormer in unmanned aerial vehicle (UAV) videos, we also evaluate STDFormer on VisDrone2019. The results show that STDFormer achieves state-of-the-art performance on VisDrone2019, which proves that it can handle small-scale object associations in UAV videos well. The code is available at https://github.com/Xiaotong-Zhu/STDFormer.

show abstract

“…Most recent multi object tracking advances focus on improving overall metrics with various detection, re-identification or association methods, while others strive for real time runtime performance and generally low computational costs. We evaluate our method in comparison to the currently fastest algorithms of the MOT17 and MOT20 benchmarks: IOU17 (Bochinski, 2017), MAA (Stadler, 2022), SORT17, SORT20 (Bewley, 2016), OCSort (Cao, 2022), CTv0 (Lohn-Jaramillo, 2022), IOU_KMM (Urbann, 2021), and PaddleDetection.…”

Section: Related Workmentioning

confidence: 99%

“…The public entries of this algorithm list 143 Hz for MOT17 and 57 Hz for MOT20. The architecture is easy to generalize and forms the foundation for various SORT inspired methods like DeepSORT (Wojke, 2017) or OCSort (Cao, 2022). OCSort is "Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking", where a observation centric filtering strategy is used.…”

Section: Related Workmentioning

confidence: 99%

Fast Timeline Based Multi Object Online Tracking

Hünermund

Groneberg

Brauckmann

2023

Transport and Telecommunication Journal

View full text Add to dashboard Cite

Fast state-of-the-art multi-object-tracking (MOT) schemes, such as reported in challenges MOT16 and Mot20, perform tracking on a single sensor, often couple tracking and detection, support only one kind of object representation or don’t take varying latencies and update rates into account. We propose a fast generic MOT system for use in real world applications which is capable of tracking objects from different sensor / detector types with their respective latencies and update rates. An SORT inspired online tracking scheme is extended by time awareness using timelines as unifying concept. The system supports different object, sensor and filter and tracking types by modularizing and generalizing the online tracking scheme, while ensuring high performance using an efficient data-oriented C++-template-based implementation. Using the proposed system we achieve, with comparable evaluation metrics, framerates up to ten times higher than the fastest MOT schemes publicly listed for the axis-aligned bounding-box tracking challenges MOT17 and MOT20.

show abstract

Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking

Cited by 33 publications

References 38 publications

StrongSORT: Make DeepSORT Great Again

StrongSORT: Make DeepSORT Great Again

STDFormer: Spatial-Temporal Motion Transformer for Multiple Object Tracking

Fast Timeline Based Multi Object Online Tracking

Contact Info

Product

Resources

About