Bayesian fusion of thermal and visible spectra camera data for region based tracking with rapid background adaptation

Abstract

This paper presents a method for optimally combining pixel information from an infra-red thermal imaging camera, and a conventional visible spectrum colour camera, for tracking a moving target. The tracking algorithm rapidly re-learns its background models for each camera modality from scratch at every frame. This enables, firstly, automatic adjustment of the relative importance of thermal and visible information in decision making, and, secondly, a degree of "camouflage target" tracking by continuously re-weighting the importance of those parts of the target model that are most distinct from the present background at each frame. Furthermore, this very rapid background adaptation ensures robustness to large, sudden and arbitrary camera motion, and thus makes this method a useful tool for robotics, for example visual servoing of a pan-tilt turret mounted on a moving robot vehicle. The method can be used to track any kind of arbitrarily shaped or deforming object, however the combination of thermal and visible information proves particularly useful for enabling robots to track people. The method is also important in that it can be readily extended for data fusion of an arbitrary number of statistically independent features from one or arbitrarily many imaging modalities.

A review of algorithms, methods, and techniques for detecting UAVs and UAS using audio, radiofrequency, and video applications

“…Many color palettes are available to map these temperature measurements generating different brightness and contrast, that can be used by a linear transfer function that can be seen as a sliding window to change the location and width [43]. According to [44], the fusion of infrared thermal images with the visible spectrum is useful to detect objects with temperature differentials due to emission or reflection, such as drones and people. A system of this kind that uses conventional sensors captures objective information such as emitted and reflected radiation.…”

Distractor-Supported Single Target Tracking in Extremely Cluttered Scenes

“…A unified model to select the best matching metric (attribution selection) and most stable sub-region of the target (spatial selection) for tracking was proposed in [12]. Hong et al [13] learned a discriminative (matching) metric that adaptively computed the importance of different features, and online adaptive attribute weighting was also proposed in [14][15][16]. Posseger et al [17] recently proposed a distractor-aware target model to select salient colours in single target tracking.…”

“…Posseger et al [17] recently proposed a distractor-aware target model to select salient colours in single target tracking. However, none of the methods [12][13][14][15][16][17] actively searches and memorises the trajectories of the distractors in scenes, or exploits a global dynamic constraint to improve single target tracking. In addition, our paper addresses video sequences that are so extreme that both target and distractors may have identical appearance, and cannot be disambiguated by any appearance features.…”

Two Algorithms for the Detection and Tracking of Moving Vehicle Targets in Aerial Infrared Image Sequences

“…In the field of infrared target tracking, good results have been obtained in previous research using region-based [15,16], contour-based [17,18], model-based [19,20] and feature-based [21,22] algorithms. For example, Ling et al [23] defined the evaluation criterion for the tracking effect and searched for the relatively accurate region similar to the reference region by maximizing the eigenvalues of the covariance matrix of the local complexity when the tracking error was large.…”