Channel Exchanging for RGB-T Tracking

Abstract

It is difficult to achieve all-weather visual object tracking in an open environment only utilizing single modality data input. Due to the complementarity of RGB and thermal infrared (TIR) data in various complex environments, a more robust object tracking framework can be obtained using video data of these two modalities. The fusion methods of RGB and TIR data are the core elements to determine the performance of the RGB-T object tracking method, and the existing RGB-T trackers have not solved this problem well. In order to solve the current low utilization of information intra single modality in aggregation-based methods and between two modalities in alignment-based methods, we used DiMP as the baseline tracker to design an RGB-T object tracking framework channel exchanging DiMP (CEDiMP) based on channel exchanging. CEDiMP achieves dynamic channel exchanging between sub-networks of different modes hardly adding any parameters during the feature fusion process. The expression ability of the deep features generated by our data fusion method based on channel exchanging is stronger. At the same time, in order to solve the poor generalization ability of the existing RGB-T object tracking methods and the poor ability in the long-term object tracking, more training of CEDiMP on the synthetic dataset LaSOT-RGBT is added. A large number of experiments demonstrate the effectiveness of the proposed model. CEDiMP achieves the best performance on two RGB-T object tracking benchmark datasets, GTOT and RGBT234, and performs outstandingly in the generalization testing.

Efficient RGB-T Tracking via Cross-Modality Distillation

“…To evaluate the superiority of our proposed method, we compare our method with some existing state-of-the-art RGB-T trackers, including MANet [17], DAFNet [6], Net [39], TODA [28], MACNet [30], CAT [12], CEDiMP [38], SiamCDA [36], mfDiMP [31], FANet [40], CBPNet [27], MANet++ [33], JMMAC [32], ADRNet [33], TFNet [41], MFGNet [25], M5LNet [23], HMFT [34], APFNet [26] and MFNet [35], on three challenging datasets. Considering that existing methods usually employ different training datasets, we use the LasHeR training set to retrain some of these algorithms for fair comparisons, including FANet * , DAFNet * , MANet * and mfDiMP * .…”