Two Streams Multiple-Model Object Tracker for Thermal Infrared Video

Abstract

Thermal infrared (TIR) visual object tracking has been applied in various applications, such as pedestrian detection, wildlife observation, surveillance systems, and so on. The tracker function is to track a particular object of interest and generate its trajectory which will be integrated into a decision making process. Many trackers that are based on fully convolutional neural networks (CNNs) have good performance for RGB input, but it is not the case for the TIR input. Its lack of texture information and the fact that it produces similar heat maps between two nearby objects make the tracking task very challenging. By relying on a fully CNN network alone, a tracker can learn the appearance model but it will not work well if the object heat map looks too similar to the background. Hence, a Siamese CNN network can be implemented to complement the fully CNN, as it allows a set of recent object templates to be used for matching purposes. Yet, the Siamese network alone is not accurate especially in the case of occlusions, as the stored templates rarely produce robust matching. Thus, we propose a two-stream CNN tracker that combines the fully CNN and the Siamese CNN such that each network keeps a set of matching models to cater to diverse appearance changes. Furthermore, the CNN layers are shared between both CNN streams to reduce computational burden. A single dense score map is produced by overlaying the normalized scores of the two streams. The experiments on VOT-TIR 2016 database show that our tracker works well for the datasets with high motion blur, occlusion, and appearance deformation. Besides, the Siamese CNN response map can also be used as an indicator to decide the size of the search region. INDEX TERMS Siamese networks, single object tracking, thermal infrared video.

Deep-learning based single object tracker for night surveillance

“…It will update the CNN models in the branches which has most similar appearance with the current target estimation. In [3,13], a general tracking framework for thermal infrared videos has been proposed. Thermal images exhibit similar properties to night surveillance images where the target object usually consists of low contrast information and negligable textures.…”

“…During network updates, parent nodes will be replaced by the new node so that there is no redundancy in the pool of target object appearance models. In [13], a Siamese approach is utilized in which pair of patches are compared to find the most likely location of the target object in the current frame. Figure 2 illustrates the overall workflow of the proposed tracking methodology.…”

“…Note that since the gradient values are all positive, the accumulated sum , will keep increasing during the training process which will cause the learning rate in (13) to shrink and eventually become infinitesimally small. At this point, the optimizer is not able to learn any new knowledge.…”

“…An example of basic CNN network [12] Due to the CNN capability, this paper proposes a method of online tracking of object of interest for night surveillance application through deep-learning approach. A network with 3 convolutional layers and 3 fully connected layers is used to model the object appearance as proposed in [13]. The fully-connected layers will be updated online to cater the changes in target object appearance as it moves around the scene under different lighting conditions.…”

COVID-19 Screening Using a Lightweight Convolutional Neural Network with Generative Adversarial Network Data Augmentation

“…The proposed algorithm, which we termed LightCovidNet can be categorized as a compact architecture compared to other deep networks such as [15,16] that have more than 100 layers of convolutional neural networks (CNN). Compact networks such as in [17,18] are suitable for the situations wherein the training data are limited, such as in the case of chest X-ray images of the COVID-19 patients.…”