Object Tracking in Hyperspectral Videos with Convolutional Features and Kernelized Correlation Filter

Abstract

Target tracking in hyperspectral videos is a new research topic. In this paper, a novel method based on convolutional network and Kernelized Correlation Filter (KCF) framework is presented for tracking objects of interest in hyperspectral videos. We extract a set of normalized three-dimensional cubes from the target region as fixed convolution filters which contain spectral information surrounding a target. The feature maps generated by convolutional operations are combined to form a three-dimensional representation of an object, thereby providing effective encoding of local spectral-spatial information. We show that a simple two-layer convolutional networks is sufficient to learn robust representations without the need of offline training with a large dataset. In the tracking step, KCF is adopted to distinguish targets from neighboring environment. Experimental results demonstrate that the proposed method performs well on sample hyperspectral videos, and outperforms several state-of-the-art methods tested on grayscale and color videos in the same scene.

Object Tracking in Hyperspectral-Oriented Video with Fast Spatial-Spectral Features

“…These two methods may lose valuable information and are computationally expensive. Qian et al [37] extracts features using the 3D patches selected from an object area in the first frame, but the correlations among bands were neglected. Xiong et al [28] proposed a spectral-spatial histogram of multi-dimensional gradients and fractional abundances of constituted material components as the object features for tracking.…”

“…As for occlusion, out-of-view and scale variation attributes, SSCF also performs better, which suggests that spatial-spectral representation is more effective in dealing with scale variation and occlusion compared to RGB representation. 36), (f) out-of-plane rotation (7), (g) out of view (4), (h) scale variation (37). The values in parentheses indicate the number of sequences associated with each attribute.…”

“…Figure 14. Success plots over eight tracking attributes, including (a) background clutter (24), (b) deformation (18), (c) illumination variation (20), (d) low resolution(27), (e) occlusion (36), (f) out-of-plane rotation (7), (g) out of view (4), (h) scale variation(37). The values in parentheses indicate the number of sequences associated with each attribute.…”

Material Based Object Tracking in Hyperspectral Videos

“…We also compared our method with two recent hyperspectral trackers, CNHT [30] and DeepHKCF [29]. Both CNHT and DeepHKCF are based on KCF but use different features.…”

“…Alternatively, Uzkent et al [29] proposed a deep kernelized correlation filter based method (DeepHKCF) for aerial object tracking at the sacrifice of valuable spectral information, in which an HSI was converted to false-color image before passing to a deep convolutional neural network. Qian et al [30] selected a set of patches as convolutional kernels for each band to extract features, but the correlations among bands were neglected. It is known that local structure inside an object region facilitates exploiting material information [16], [31].…”