Challenge-Aware RGBT Tracking

Li, Chenglong; Liu, Lei; Lu, Andong; Ji, Qing; Tang, Jin

doi:10.1007/978-3-030-58542-6_14

Cited by 83 publications

(58 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…inter-modal and temporally for intra-modal by cross-modal attention. A multi-branch architecture is employed in CAT [54] for target appearance modelling in allusion to the modalityspecific and modality-shared challenges, which totally count to 5. Unlike DAPNet [8] and DAFNet [49] keep only the fused features retained, TFNet [55] designs a trident architecture for better excavating the modality-specific information.…”

Section: Tracking With Multiple Modalitiesmentioning

confidence: 99%

Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking

Tang¹,

Xu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

We address the problem of multi-modal object tracking in video and explore various options of fusing the complementary information conveyed by the visible (RGB) and thermal infrared (TIR) modalities including pixel-level, feature-level and decision-level fusion. Specifically, different from the existing methods, paradigm of image fusion task is heeded for fusion at pixel level. Featurelevel fusion is fulfilled by attention mechanism with channels excited optionally. Besides, at decision level, a novel fusion strategy is put forward since an effortless averaging configuration has shown the superiority. The effectiveness of the proposed decision-level fusion strategy owes to a number of innovative contributions, including a dynamic weighting of the RGB and TIR contributions and a linear template update operation. A variant of which produced the winning tracker at the Visual Object Tracking Challenge 2020 (VOT-RGBT2020). The concurrent exploration of innovative pixel-and feature-level fusion strategies highlights the advantages of the proposed decision-level fusion method. Extensive experimental results on three challenging datasets, i.e., GTOT, VOT-RGBT2019, and VOT-RGBT2020, demonstrate the effectiveness and robustness of the proposed method, compared to the state-of-the-art approaches. Code will be shared at https://github.com/Zhangyong-Tang/DFAT.

show abstract

Section: Tracking With Multiple Modalitiesmentioning

confidence: 99%

Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking

Tang¹,

Xu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To obtain the inner-class variance of different objects, which is critical for tracking, the two-stream multi-resolution CNN branch is applied, and the shallower layer contained more detail information, which is more discriminative [25]. And the target appearance under different challenges could be well represented in different layers.…”

Section: Overviewmentioning

confidence: 99%

CMC2R: Cross‐modal collaborative contextual representation for RGBT tracking

Liu

Luo²,

Yan

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

The key challenge in RBGT tracking is how to fuse dual-modality information to build a robust RGB-T tracker. Motivated by CNN structure for local features, and visual transformer structure for global representations, the authors propose a two-stream hybrid structure, termed CMC 2 R, to take advantage of convolutional operations and self-attention mechanisms to lean the enhanced representation. CMC 2 R fuses local features and global representations under different resolutions through the transformer layer of the encoder block, and the two modalities are collaborated to get contextual information by the spatial and channel self-attention. The temporal association is performed with the track query, each track query models the entire track of an object, and updated frame-by-frame to build the long-range temporal relation. Experimental results show the effectiveness of the proposed method, and achieve the SOTAs performance.

show abstract

“…TFNet [27] deploys a trident branch architecture and each branch is specific for the RGB, TIR and fused features. Different from the most existing RGBT trackers, CAT [28], ADRNet [29] and APFNet [30] make their network construction more concrete for both modality-specific (e.g., illumination variation in RGB and thermal crossover in TIR) and modality-shared (e.g., scale variation) challenges. In CAT [28], all the challenge-specific features are adaptively aggregated and then complemented to the basic learning procedure of both modalities.…”

Section: A Mdnet-based Rgbt Trackersmentioning

confidence: 99%

“…Different from the most existing RGBT trackers, CAT [28], ADRNet [29] and APFNet [30] make their network construction more concrete for both modality-specific (e.g., illumination variation in RGB and thermal crossover in TIR) and modality-shared (e.g., scale variation) challenges. In CAT [28], all the challenge-specific features are adaptively aggregated and then complemented to the basic learning procedure of both modalities. ADRNet [29] designs an attribute-driven residual block to measure the appearance model under different circumstance and then all the information is ensembled before the residual connection to the Fig.…”

Section: A Mdnet-based Rgbt Trackersmentioning

confidence: 99%

A Survey for Deep RGBT Tracking

Tang¹,

Xu²,

Wu³

2022

Preprint

View full text Add to dashboard Cite

Visual object tracking with the visible (RGB) and thermal infrared (TIR) electromagnetic waves, shorted in RGBT tracking, recently draws increasing attention in the tracking community. Considering the rapid development of deep learning, a survey for the recent deep neural network based RGBT trackers is presented in this paper. Firstly, we give brief introduction for the RGBT trackers concluded into this category. Then, a comparison among the existing RGBT trackers on several challenging benchmarks is given statistically. Specifically, MDNet and Siamese architectures are the two mainstream frameworks in the RGBT community, especially the former. Trackers based on MDNet achieve higher performance while Siamese-based trackers satisfy the real-time requirement. In summary, since the large-scale dataset LasHeR is published, the integration of end-to-end framework, e.g., Siamese and Transformer, should be further considered to fulfil the real-time as well as more robust performance. Furthermore, the mathematical meaning should be more considered during designing the network. This survey can be treated as a look-up-table for researchers who are concerned about RGBT tracking.

show abstract

Challenge-Aware RGBT Tracking

Cited by 83 publications

References 15 publications

Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking

Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking

CMC2R: Cross‐modal collaborative contextual representation for RGBT tracking

A Survey for Deep RGBT Tracking

Contact Info

Product

Resources

About