Weighted Sparse Representation Regularized Graph Learning for RGB-T Object Tracking

Li, Chenglong; Zhao, Nan; Lu, Yijuan; Zhu, Chengli; Tang, Jin

doi:10.1145/3123266.3123289

Cited by 163 publications

(156 citation statements)

References 38 publications

Supporting

Mentioning

156

Contrasting

Order By: Relevance

“…For the next three rows, we fine-tune the feature extractor and/or IoU-Net with synthetic TIR images. In this case, [34]. We can see our mfDiMP outperforms DiMP with an absolute gain of 6.7% and 4.2% in terms of precision rate and success rate respectively.…”

Section: Implementation Detailsmentioning

confidence: 78%

See 1 more Smart Citation

Multi-Modal Fusion for End-to-End RGB-T Tracking

Zhang

Danelljan

González-García

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

125

View full text Add to dashboard Cite

We propose an end-to-end tracking framework for fusing the RGB and TIR modalities in RGB-T tracking. Our baseline tracker is DiMP (Discriminative Model Prediction), which employs a carefully designed target prediction network trained end-to-end using a discriminative loss. We analyze the effectiveness of modality fusion in each of the main components in DiMP, i.e. feature extractor, target estimation network, and classifier. We consider several fusion mechanisms acting at different levels of the framework, including pixel-level, feature-level and response-level. Our tracker is trained in an end-to-end manner, enabling the components to learn how to fuse the information from both modalities. As data to train our model, we generate a largescale RGB-T dataset by considering an annotated RGB tracking dataset (GOT-10k) and synthesizing paired TIR images using an image-to-image translation approach. We perform extensive experiments on VOT-RGBT2019 dataset and RGBT210 dataset, evaluating each type of modality fusing on each model component. The results show that the proposed fusion mechanisms improve the performance of the single modality counterparts. We obtain our best results when fusing at the feature-level on both the IoU-Net and the model predictor, obtaining an EAO score of 0.391 on VOT-RGBT2019 dataset. With this fusion mechanism we achieve the state-of-the-art performance on RGBT210 dataset.

show abstract

Section: Implementation Detailsmentioning

confidence: 78%

“…RGBT210 dataset [34] contains 210 highly-aligned public RGB and TIR video pairs for testing, with 210K frames in total and a maximum of 8K frames per sequence pair.…”

Section: Evaluation Datasets and Protocolsmentioning

confidence: 99%

Multi-Modal Fusion for End-to-End RGB-T Tracking

Zhang

Danelljan

González-García

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

125

View full text Add to dashboard Cite

show abstract

“…And each frame pair is annotated with ground truth bounding box. RGBT234 dataset is a large-scale RGBT tracking dataset extended from RGBT210 dataset [30]. It contains 234 RGBT videos and each video has a RGB video and a thermal video.…”

Section: Evaluation Settingmentioning

confidence: 99%

“…We further compare our tracker with several state-of-the-art RGBT trackers, including RT-MDNet+RGBT, DAT+RGBT, SiamDW+RGBT, CSR [5], JSR [41], L1-PF [45], SGT [30], MDNet+RGBT1, and MDNet+RGBT2. Since there are few RGBT trackers [26,27,30,33], some RGB tracking methods have been extended to RGB-T ones by concatenating RGB and thermal features into a single vector or regarding the thermal as an extra channel, such as RT-MDNet, DAT, SiamDW and CFnet. Figure 3(b) shows that our tracker significantly outperforms them, demonstrating the effectiveness of employing RGB and thermal information adaptively to construct robust feature representations in our approach.…”

Section: Evaluation On Gtotmentioning

confidence: 99%

“…Recently, it has received increasing attention as it is able to achieve robust tracking performance in challenging environments by utilizing inter-modal complementarity. Despite much progress in RGBT tracking [4,[29][30][31], there are still many problems need to be solved, where how to effectively fuse RGB and thermal infrared sources is a core factor in boosting tracking performance and still not solved well.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dense Feature Aggregation and Pruning for RGBT Tracking

Zhu

Luo

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

Self Cite

148

106

View full text Add to dashboard Cite

How to perform effective information fusion of different modalities is a core factor in boosting the performance of RGBT tracking. This paper presents a novel deep fusion algorithm based on the representations from an end-to-end trained convolutional neural network. To deploy the complementarity of features of all layers, we propose a recursive strategy to densely aggregate these features that yield robust representations of target objects in each modality. In different modalities, we propose to prune the densely aggregated features of all modalities in a collaborative way. In a specific, we employ the operations of global average pooling and weighted random selection to perform channel scoring and selection, which could remove redundant and noisy features to achieve more robust feature representation. Experimental results on two RGBT tracking benchmark datasets suggest that our tracker achieves clear stateof-the-art against other RGB and RGBT tracking methods.

show abstract

Learning Collaborative Sparse Correlation Filter for Real-Time Multispectral Object Tracking

Wang

Tang

et al. 2018

Advances in Brain Inspired Cognitive Systems

View full text Add to dashboard Cite

Weighted Sparse Representation Regularized Graph Learning for RGB-T Object Tracking

Cited by 163 publications

References 38 publications

Multi-Modal Fusion for End-to-End RGB-T Tracking

Multi-Modal Fusion for End-to-End RGB-T Tracking

Dense Feature Aggregation and Pruning for RGBT Tracking

Learning Collaborative Sparse Correlation Filter for Real-Time Multispectral Object Tracking

Contact Info

Product

Resources

About