Deep Learning of Appearance Models for Online Object Tracking

Abstract

This paper introduces a novel deep learning based approach for vision based single target tracking. We address this problem by proposing a network architecture which takes the input video frames and directly computes the tracking score for any candidate target location by estimating the probability distributions of the positive and negative examples. This is achieved by combining a deep convolutional neural network with a Bayesian loss layer in a unified framework. In order to deal with the limited number of positive training examples, the network is pre-trained offline for a generic image feature representation and then is fine-tuned in multiple steps. An online fine-tuning step is carried out at every frame to learn the appearance of the target. We adopt a two-stage iterative algorithm to adaptively update the network parameters and maintain a probability density for target/non-target regions. The tracker has been tested on the standard tracking benchmark and the results indicate that the proposed solution achieves state-of-the-art tracking results.

Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies

“…Other approaches use covariance matrix representation, pixel comparison representation, SIFT-like features, or pose features [25,83,29,22,50]. Recently, deep neural network architectures have been used for modeling appearance [21,36,84]. In these architectures, high-level features are extracted by convolutional neural networks trained for a specific task.…”

End-to-End Representation Learning for Correlation Filter Based Tracking

“…The ImageNet+CF variant employs features taken from a network trained to solve the ImageNet classification challenge [28]. The results show that these features, which are often the first choice for combining CFs with CNNs [7,9,22,26,32,36], are significantly worse than those learned by CFNet and the Baseline experiment. The particularly poor performance of these features at deeper layers is somewhat unsurprising, since these layers are expected to have greater invariance to position when trained for classification.…”

“…The simplest approach is to disregard the lack of a-priori knowledge and adapt a pre-trained deep convolutional neural network (CNN) to the target, for example by using stochastic gradient descent (SGD), the workhorse of deep network optimization [32,26,36]. The extremely limited training data and large number of parameters make this a difficult learning problem.…”

“…This problem emerges naturally in applications such as visual object tracking, where the goal is to re-detect an object over a video with the sole supervision of a bounding box at the beginning of the sequence. The main challenge is the lack of a-priori knowledge of the target object, which can be of any class.The simplest approach is to disregard the lack of a-priori knowledge and adapt a pre-trained deep convolutional neural network (CNN) to the target, for example by using stochastic gradient descent (SGD), the workhorse of deep network optimization [32,26,36]. The extremely limited training data and large number of parameters make this a difficult learning problem.…”

Eliminating Exposure Bias and Metric Mismatch in Multiple Object Tracking

“…Many approaches rely on appearance [17,29,57,59,11,33,47], motion [13], or social cues [20,44]. They are mostly used to associate pairs of detections, and only account for very short-term correlations.…”