A General Framework for Combining Visual Trackers – The "Black Boxes" Approach

Abstract

Abstract.Over the past few years researchers have been investigating the enhancement of visual tracking performance by devising trackers that simultaneously make use of several different features. In this paper we investigate the combination of synchronous visual trackers that use different features while treating the trackers as "black boxes". That is, instead of fusing the usage of the different types of data as has been performed in previous work, the combination here is allowed to use only the trackers' output estimates, which may be modified before their propagation to the next time step. We propose a probabilistic framework for combining multiple synchronous trackers, where each separate tracker outputs a probability density function of the tracked state, sequentially for each image. The trackers may output either an explicit probability density function, or a sample-set of it via CONDENSATION. Unlike previous tracker combinations, the proposed framework is fairly general and allows the combination of any set of trackers of this kind, even in different state-spaces of different dimensionality, under a few reasonable assumptions. The combination may consist of different trackers that track a common object, as well as trackers that track separate, albeit related objects, thus improving the tracking performance of each object. The benefits of merely using the final estimates of the separate trackers in the combination are twofold. Firstly, the framework for the combination is fairly general and may be easily used from the software aspects. Secondly, the combination may be performed in a distributed setting, where each separate tracker runs on a different site and uses different data, while avoiding the need to share the data. The suggested framework was successfully tested using various state-spaces and datasets, demonstrating that fusing the trackers' final distribution estimates may indeed be applicable.

Disagreement-Based Multi-system Tracking

“…A number of tracking methods have been proposed to perform fusion [13,14,18,19,16,15,17]. Different from [13,17] where multiple parts were tracked and correlated, we deal with a single target.…”

“…In [14,16] multiple trackers were fused but these trackers represent different features and they were directly combined. In [18] the tracking approach was combined via the weighted combination of the PDFs. Different from [18], our method does not perform direct multiplication but seeks a balance between the PDF of one tracker and the degree of agreement by the other trackers; also, in our method, each tracker performs prediction separately maintaining certain independence and patches at the agreed positions can be recommended to update the other trackers.…”

Fusing Time-of-Flight Depth and Color for Real-Time Segmentation and Tracking

“…Improvements have been made by adding edge information, a Kalman prediction step [6] or modifying the kernel applied to the image samples [2], but they still have the limitation that these measures rely on the same color data. Therefore, it is desireable to add additional information to the problem at hand [12] in order to get an improvement, and thus fusing time-of-flight depth data with color can lead to much better results.…”

A Probabilistic Approach to Integrating Multiple Cues in Visual Tracking

“…One important issue is how to model the dependence between different cues, which in turn determines the manner in which the cues are combined. Many methods assume that the cues are conditionally independent [7][8][9][10], while more sophisticated methods model existing dependencies explicitly by e.g. graphical models [11].…”