Multi-Task Driven Feature Models for Thermal Infrared Tracking

Abstract

Existing deep Thermal InfraRed (TIR) trackers usually use the feature models of RGB trackers for representation. However, these feature models learned on RGB images are neither effective in representing TIR objects nor taking fine-grained TIR information into consideration. To this end, we develop a multi-task framework to learn the TIR-specific discriminative features and fine-grained correlation features for TIR tracking. Specifically, we first use an auxiliary classification network to guide the generation of TIR-specific discriminative features for distinguishing the TIR objects belonging to different classes. Second, we design a fine-grained aware module to capture more subtle information for distinguishing the TIR objects belonging to the same class. These two kinds of features complement each other and recognize TIR objects in the levels of inter-class and intra-class respectively. These two feature models are learned using a multi-task matching framework and are jointly optimized on the TIR tracking task. In addition, we develop a large-scale TIR training dataset to train the network for adapting the model to the TIR domain. Extensive experimental results on three benchmarks show that the proposed algorithm achieves a relative gain of 10% over the baseline and performs favorably against the state-of-the-art methods. Codes and the proposed TIR dataset are available at https://github.com/QiaoLiuHit/MMNet.

SiamCDA: Complementarity- and Distractor-Aware RGB-T Tracking Based on Siamese Network

“…Considering the existence of large-scale thermal datasets [52], generating synthetic RGB data from thermal data can also further expand the training data. Moreover, the real thermal data is helpful for the model to learn the effective feature representation of thermal images.…”

Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences

“…This solution is able to help correct temporary missrecognitions that occur when the detector fails as well as reduce false detections. The authors in [36] try to solve the data association and tracking problem in thermal images using deep network architectures. They propose a feature model comprising of thermal infrared specific features and correlation features for thermal infrared object representation.…”

An End-to-End Dehazing Siamese Region Proposal Network for High Robustness Object Tracking

“…Gang [32] et al incorporated low-rank sparse and adaptive dictionary learning template with the classical particle filter algorithm to solve the difficulty of vehicle tracking in haze scenes. Another solution to cope with severe weather is to perform thermal infrared tracking [33], such as MCFTS [34], MLSSNet [35], MMNet [36] proposed by Liu et al In some object detection tasks [37]- [40], researchers proposed a series of pedestrian and vehicle detection methods for the haze environment. Our DH-SiamRPN provides a new fusion algorithm to solve the object tracking problem in this scenario.…”