Semantic‐guided fusion for multiple object tracking and RGB‐T tracking

Abstract

The attention mechanism has produced impressive results in object tracking, but for a good trade‐off between performance and efficiency, CNN‐based approaches still dominate, owing to quadratic complexity of attention. Here, the SGF module is proposed, an efficient feature fusion block for effective object tracking with reduced linear complexity of attention. The proposed method fuses feature with attention in a coarse‐to‐fine manner. In the low‐resolution semantic branch, the top K regions with highest attention scores are selected; in the high‐resolution detail branch, attention is only calculated within regions corresponding to the top K regions. Thus, the features from the high‐resolution branch can be efficiently fused under the guidance of low‐resolution branch. Experiments on RGB and RGB‐T datasets with reformed FairMOT and MDNet+RGBT trackers demonstrated the effectiveness of the proposed method.