VTST: Efficient Visual Tracking With a Stereoscopic Transformer

Gu, Fengwei; Lu, Jun; Cai, Chengtao; Zhu, Qidan; Ju, Zhaojie

doi:10.1109/tetci.2024.3360303

Cited by 4 publications

(1 citation statement)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the realm of visual tracking, certain works have stood out due to their focus on refining the attention mechanisms in Transformers 8 , 9 , 13 – 17 . Most of these trackers excel in managing lengthy sequential relationships and capturing comprehensive global information, yet they tend to overlook target-specific details within the designated search region.…”

Section: Related Workmentioning

confidence: 99%

Adaptive sparse attention-based compact transformer for object tracking

Pan,

Zhao,

Wang

2024

Sci Rep

View full text Add to dashboard Cite

The Transformer-based Siamese networks have excelled in the field of object tracking. Nevertheless, a notable limitation persists in their reliance on ResNet as backbone, which lacks the capacity to effectively capture global information and exhibits constraints in feature representation. Furthermore, these trackers struggle to effectively attend to target-relevant information within the search region using multi-head self-attention (MSA). Additionally, they are prone to robustness challenges during online tracking and tend to exhibit significant model complexity. To address these limitations, We propose a novel tracker named ASACTT, which includes a backbone network, feature fusion network and prediction head. First, we improve the Swin-Transformer-Tiny to enhance its global information extraction capabilities. Second, we propose an adaptive sparse attention (ASA) to focus on target-specific details within the search region. Third, we leverage position encoding and historical candidate data to develop a dynamic template updater (DTU), which ensures the preservation of the initial frame’s integrity while gracefully adapting to variations in the target’s appearance. Finally, we optimize the network model to maintain accuracy while minimizing complexity. To verify the effectiveness of our proposed tracker, ASACTT, experiments on five benchmark datasets demonstrated that the proposed tracker was highly comparable to other state-of-the-art methods. Notably, in the GOT-10K1 evaluation, our tracker achieved an outstanding success score of 75.3% at 36 FPS, significantly surpassing other trackers with comparable model parameters.

show abstract

Section: Related Workmentioning

confidence: 99%

Adaptive sparse attention-based compact transformer for object tracking

Pan,

Zhao,

Wang

2024

Sci Rep

View full text Add to dashboard Cite

show abstract

A framework of specialized knowledge distillation for Siamese tracker on challenging attributes

Li,

Shimada,

Minematsu

et al. 2024

Machine Vision and Applications

View full text Add to dashboard Cite

Enhance the 3D Object Detection With 2D Prior

Liu

2024

IEEE Access

View full text Add to dashboard Cite

In the modern field of multi-view 3D detection, interest has seen a notable rise. Existing methods are predominantly focused on constructing dense BEV (Bird's Eye View) features or utilizing sparse queries for detection. In this paper, we synergize the methodologies of sparse queries with dense BEV feature representation and integrate robust 2D detection capabilities to propose an innovative structure. Specifically, we leverage the results of 2D detection and their predicted depth to generate a series of sparse queries endowed with strong semantic and positional priors. These queries demonstrate enhanced capability in capturing challenging cases in 3D detection. Additionally, we employ 2D detection and depth information to discern critical foreground BEV queries. This strategy enables a efficient and noiseless feature aggregation, leveraging multi-view image features. Our methodology extends to a comprehensive global modeling of all BEV queries, ensuring the derivation of premium quality BEV features. Furthermore, we enhance BEV features through temporal modeling that differentiates between static and dynamic objects. Our proposed approach achieves state-of-the-art results on the nuScenes test set with 60.4%mAP and 51.7%NDS. Code will be available.INDEX TERMS Multi-view 3D object detection, 2D detection prior, Autonomous driving.

show abstract

VTST: Efficient Visual Tracking With a Stereoscopic Transformer

Cited by 4 publications

References 71 publications

Adaptive sparse attention-based compact transformer for object tracking

Adaptive sparse attention-based compact transformer for object tracking

A framework of specialized knowledge distillation for Siamese tracker on challenging attributes

Enhance the 3D Object Detection With 2D Prior

Contact Info

Product

Resources

About