Learning to track with multiple observers

Abstract

We propose a novel approach to designing algorithms for object tracking based on fusing multiple observation models. As the space of possible observation models is too large for exhaustive on-line search, this work aims to select models that are suitable for a particular tracking task at hand. During an off-line training stage observation models from various off-the-shelf trackers are evaluated. From this data different methods of fusing the observers on-line are investigated, including parallel and cascaded evaluation. Experiments on test sequences show that this evaluation is usefulfor automatically designing and assessing algorithms for a particular tracking task. Results are shown for face tracking with a handheld camera and hand tracking for gesture interaction. We show that for these cases combining a small number of observers in a sequential cascade results in efficient algorithms that are both robust and precise.

Ensemble Classifier for Combining Stereo Matching Algorithms

“…Motten et al [27] presented a decision tree classifier implemented on FPGA that selects among multiple disparity hypotheses generated by trinocular stereo. Stenger et al [35] select trackers either based on maximum confidence or by forming a cascade and deciding whether to accept the current tracker or proceed to the next one, hence avoiding the execution of all trackers a priori. Gao et al [9] proposed a method for forming an ensemble of trackers that considers the reliability of each tracker individually and also the correlations of pairs of trackers.…”

Motion Models that Only Work Sometimes

“…They use a mixed dynamic model combining constant velocity for position and Brownian evolution for size.Čehovin et al [31] built on that approach with a constellation of local visual parts, which get resampled spatially based on consistency with an object's higher-level appearance. Most relevant to our supervised learning-based approach, Stenger et al [30] assess the suitability of different measurement models for a particular tracking scenario, e.g. face or hand tracking.…”

Disagreement-Based Multi-system Tracking

“…Different from [18], our method does not perform direct multiplication but seeks a balance between the PDF of one tracker and the degree of agreement by the other trackers; also, in our method, each tracker performs prediction separately maintaining certain independence and patches at the agreed positions can be recommended to update the other trackers. In [20], the tracking combination method is trained for specific scenarios. Different from [20], our method is based on the disagreement-based semi-supervised learning and do not require an off-line training process; also, it can be applied to general videos, and performs very well on a fairly large benchmark dataset.…”

“…In [20], the tracking combination method is trained for specific scenarios. Different from [20], our method is based on the disagreement-based semi-supervised learning and do not require an off-line training process; also, it can be applied to general videos, and performs very well on a fairly large benchmark dataset. In [21], mutual information was used for the fusion.…”