SparseTT: Visual Tracking with Sparse Transformers

Fu, Zhaohui; Fu, Zehua; Liu, Qingjie; Cai, Wenrui; Wang, Yunlong

doi:10.48550/arxiv.2205.03776

Cited by 6 publications

(14 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the model architecture, feature extraction, and feature integration techniques, recent deep trackers can be classified as three categories: CNN-based trackers [29,88,89,90,31,91,32,92,93,34,33,94], CNN-Transformer based trackers [46,47,48,49,50,51,52,53,54,55,56,57] and fully-Transformer based trackers [58,59,60,61,62,63,64]. CNN-based trackers rely solely on a CNN architecture for feature extraction and target detection, while CNN-Transformer based trackers and fully-Transformer based trackers partially and fully rely on a Transformer architecture, respectively.…”

Section: Transformer In Single Object Trackingmentioning

confidence: 99%

“…TrDiMP [46], DTT [50], HiFT [51], TransT [47], STARK [48], ToMP [52], CSWinTT [56], AiATrack [57], SiamTPN [53], UTT [54], TrTr [49], CTT [55] Fully-Transformer based Trackers Two-stream Two-stage Trackers DualTFR [58], Swin-Track [59], SparseTT [60] One-stream One-Stage Trackers MixFormer [61], SimTrack [62], OSTrack [63], ProContEXT [64] features are extracted using the ResNet50 [97] model. Then these features are reshaped using a 1 × 1 convolutional layer and fed to the feature fusion network.…”

Section: Transformers In Vot Cnn-transformer Based Trackersmentioning

confidence: 99%

“…Based on the tracking accuracy, SwinTrack out- While the self-attention mechanism of the Transformer captures long-range dependencies, it may not adequately capture target-specific cues, making it susceptible to distraction by background clutter. Fu et al [60] noticed this issue and then proposed a tracker, referred as SparseTT, by using a spare attention module to provide the target-specific attention in the search region features. In this tracker, the Swin Transformer is used to extract the target template and search region features and then fed them to a sparse Transformer.…”

Section: Transformers In Vot Cnn-transformer Based Trackersmentioning

confidence: 99%

“…17. The fully-Transformer based trackers: Mix-Former [61], SparseTT [60], and ProContEXT [64] showed better performance than the CNN-based trackers in many challenging attributes. In particular, almost all of the fully Transformer-based trackers successfully handled the fast motion (FM) and out-of-view (OV) scenarios and outperformed the CNN-based trackers by a large margin, due to their long-range feature capturing capability.…”

Section: Analysis On Otb100 Datasetmentioning

confidence: 99%

“…For the last two years, due to the great success of Transformers in other computer vision tasks, a few transformer based VOT approaches [46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64] have been proposed and showed excellent performance on publicly available single object tracking benchmark datasets. They have been proposed as discriminative or Siamese-based trackers and showed outstanding performances than the CNN-based trackers due to their long range feature capturing techniques and high discriminative capabilities.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Transformers in Single Object Tracking: An Experimental Survey

Thangavel¹,

Thanikasalam²,

Ramanan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Single object tracking is a well-known and challenging research topic in computer vision. Over the last two decades, numerous researchers have proposed various algorithms to solve this problem and achieved promising results. Recently, Transformer-based tracking approaches have ushered in a new era in single object tracking due to their superior tracking robustness. Although several survey studies have been conducted to analyze the performance of trackers, there is a need for another survey study after the introduction of Transformers in single object tracking. In this survey, we aim to analyze the literature and performances of Transformer tracking approaches. Therefore, we conduct an in-depth literature analysis of Transformer tracking approaches and evaluate their tracking robustness and computational efficiency on challenging benchmark datasets. In addition, we have measured their performances on different tracking scenarios to find their strength and weaknesses. Our survey provides insights into the underlying principles of Transformer tracking approaches, the challenges they face, and their future directions.

show abstract

Section: Transformer In Single Object Trackingmentioning

confidence: 99%

Section: Transformers In Vot Cnn-transformer Based Trackersmentioning

confidence: 99%

Section: Transformers In Vot Cnn-transformer Based Trackersmentioning

confidence: 99%

Section: Analysis On Otb100 Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Transformers in Single Object Tracking: An Experimental Survey

Thangavel¹,

Thanikasalam²,

Ramanan³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Discriminative correlation filters for visual object tracking with adaptive channel-based sample weights

孙,

junqi,

professor

et al. 2024

Speckle 2023: VIII International Conference on Speckle Metrology

View full text Add to dashboard Cite

In order to improve the accuracy and robustness of correlation filtering algorithm in more challenging scenarios, a target tracking algorithm based on adaptive channel sample weights is proposed in this paper. Firstly, channel attention modules are added to sample branch and search branch respectively to weight the features of each branch respectively, which can effectively improve the feature expression ability. Secondly, according to the channel weights of the two branches, we learn to fuse the weight information, interact the features of the two branches, and suppress the background information. Finally, the sample learning weights are reassigned according to the channel weights of the history frame to make more efficient use of the background information. In this paper, the OTB100 data set is used to verify the effectiveness of the improved algorithm. The area under the curve is increased by 1.6%, which proves the effectiveness of the improved algorithm.

show abstract

Scaling-Invariant Max-Filtering Enhancement Transformers for Efficient Visual Tracking

Chen,

Xiong,

Meng

et al. 2023

Preprint

View full text Add to dashboard Cite

Real-time tracking is one of the most challenging problems in computer vision. Most Transformer-based trackers usually require expensive computational and storage power, which leads to the inability of these robust trackers to achieve real-time performance in resource-constrained devices. In this work, we propose a lightweight tracker, AnteaTrack. To localize the target more precisely, we develop a scaling-invariant max-filtering operator employing a sliding window combined with local max-pooling, which filters out the suspected target from the feature and performs an augmented representation while suppressing the background. For a more tight bounding box, we employ Pixel-Shuffle to increase the resolution of the feature map and get a more fine-grained representation of the target. In addition, AnteaTrack can run in real-time at 47 frames per second(FPS) on the CPU. We tested AnteaTrack on 5 datasets, and a large number of experiments have shown that AnteaTrack provides the most efficient solution compared to the same type of CPU real-time trackers. The code will be available at https://github.com/cnchange/AnteaTrack.

show abstract

SparseTT: Visual Tracking with Sparse Transformers

Cited by 6 publications

References 0 publications

Transformers in Single Object Tracking: An Experimental Survey

Transformers in Single Object Tracking: An Experimental Survey

Discriminative correlation filters for visual object tracking with adaptive channel-based sample weights

Scaling-Invariant Max-Filtering Enhancement Transformers for Efficient Visual Tracking

Contact Info

Product

Resources

About