RGB-T object tracking: Benchmark and baseline

Li, Chenglong; Liang, Xinyan; Lu, Yijuan; Zhao, Nan; Tang, Jin

doi:10.1016/j.patcog.2019.106977

Cited by 333 publications

(224 citation statements)

References 57 publications

Supporting

Mentioning

224

Contrasting

Order By: Relevance

“…For the RGB-T target-driven global attention network, we use GTOT-50 [19] as training dataset and tracking on RGBT-234 dataset [3], and also use RGBT-234 dataset for training and test on GTOT-50 dataset. Following [4], the ground truth binary mask used for training is generated from corresponding training dataset.…”

Section: Trainingmentioning

confidence: 99%

“…Recently, some researchers resort to other data to help improve the robustness of visual trackers, such as thermal images [3], natural language description [4] and depth images [5]. Compared to text and depth images, thermal sensor is not sensitive to lighting condition and can capture the target object at far distance, and it still works well at night while RGB, depth or text may failed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Target-Oriented Dual Attention for Robust RGB-T Tracking

Yang

Zhu

Wang

et al. 2019

2019 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

RGB-Thermal object tracking attempt to locate target object using complementary visual and thermal infrared data. Existing RGB-T trackers fuse different modalities by robust feature representation learning or adaptive modal weighting. However, how to integrate dual attention mechanism for visual tracking is still a subject that has not been studied yet. In this paper, we propose two visual attention mechanisms for robust RGB-T object tracking. Specifically, the local attention is implemented by exploiting the common visual attention of RGB and thermal data to train deep classifiers. We also introduce the global attention, which is a multi-modal target-driven attention estimation network. It can provide global proposals for the classifier together with local proposals extracted from previous tracking result. Extensive experiments on two RGB-T benchmark datasets validated the effectiveness of our proposed algorithm.

show abstract

Section: Trainingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning Target-Oriented Dual Attention for Robust RGB-T Tracking

Yang

Zhu

Wang

et al. 2019

2019 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…We initialize parameters of our GA using the pre-trained model in VGG-M [30], and then fine-tune it using RGBT dataset. Note that when we conduct testing on the GTOT dataset [17], we fine-tune GA using the RGBT234 dataset [18], and vice versa. We use the stochastic gradient descent (SGD) algorithm [14] to train GA, and set the learning parameters as follows.…”

Section: Progressive Learning Algorithmmentioning

confidence: 99%

“…In this section, we will compare our MANet with state-of-the-art RGB and RGBT tracking methods on two RGBT tracking benchmark datasets, GTOT [17] and RGBT234 [18], and then evaluate the main components of MANet in detail for better understanding of our approach.…”

Section: Performance Evaluationmentioning

confidence: 99%

Multi-Adapter RGBT Tracking

Zheng

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

128

116

View full text Add to dashboard Cite

The task of RGBT tracking aims to take the complementary advantages from visible spectrum and thermal infrared data to achieve robust visual tracking, and receives more and more attention in recent years. Existing works focus on modality-specific information integration by introducing modality weights to achieve adaptive fusion or learning robust feature representations of different modalities. Although these methods could effectively deploy the modality-specific properties, they ignore the potential values of modality-shared cues as well as instance-aware information, which are crucial for effective fusion of different modalities in RGBT tracking. In this paper, we propose a novel Multi-Adapter convolutional Network (MANet) to jointly perform modality-shared, modality-specific and instance-aware feature learning in an end-to-end trained deep framework for RGBT tracking. We design three kinds of adapters within our network. In a specific, the generality adapter is to extract shared object representations, the modality adapter aims at encoding modality-specific information to deploy their complementary advantages, and the instance adapter is to model the appearance properties and temporal variations of a certain object. Moreover, to reduce computational complexity for real-time demand of visual tracking, we design a parallel structure of generic adapter and modality adapter. Extensive experiments on two RGBT tracking benchmark datasets demonstrate the outstanding performance of the proposed tracker against other state-ofthe-art RGB and RGBT tracking algorithms.

show abstract

“…It was found that thermal infrared sensors provide a more stable signal for these scenarios. Therefore, RGB-T tracking has drawn more research attention recently [31,34,32,35].…”

Section: Introductionmentioning

confidence: 99%

Multi-Modal Fusion for End-to-End RGB-T Tracking

Zhang

Danelljan

González-García

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

125

View full text Add to dashboard Cite

We propose an end-to-end tracking framework for fusing the RGB and TIR modalities in RGB-T tracking. Our baseline tracker is DiMP (Discriminative Model Prediction), which employs a carefully designed target prediction network trained end-to-end using a discriminative loss. We analyze the effectiveness of modality fusion in each of the main components in DiMP, i.e. feature extractor, target estimation network, and classifier. We consider several fusion mechanisms acting at different levels of the framework, including pixel-level, feature-level and response-level. Our tracker is trained in an end-to-end manner, enabling the components to learn how to fuse the information from both modalities. As data to train our model, we generate a largescale RGB-T dataset by considering an annotated RGB tracking dataset (GOT-10k) and synthesizing paired TIR images using an image-to-image translation approach. We perform extensive experiments on VOT-RGBT2019 dataset and RGBT210 dataset, evaluating each type of modality fusing on each model component. The results show that the proposed fusion mechanisms improve the performance of the single modality counterparts. We obtain our best results when fusing at the feature-level on both the IoU-Net and the model predictor, obtaining an EAO score of 0.391 on VOT-RGBT2019 dataset. With this fusion mechanism we achieve the state-of-the-art performance on RGBT210 dataset.

show abstract

RGB-T object tracking: Benchmark and baseline

Cited by 333 publications

References 57 publications

Learning Target-Oriented Dual Attention for Robust RGB-T Tracking

Learning Target-Oriented Dual Attention for Robust RGB-T Tracking

Multi-Adapter RGBT Tracking

Multi-Modal Fusion for End-to-End RGB-T Tracking

Contact Info

Product

Resources

About