A Novel Performance Evaluation Methodology for Single-Target Trackers

Abstract

Abstract-This paper addresses the problem of single-target tracker performance evaluation. We consider the performance measures, the dataset and the evaluation system to be the most important components of tracker evaluation and propose requirements for each of them. The requirements are the basis of a new evaluation methodology that aims at a simple and easily interpretable tracker comparison. The ranking-based methodology addresses tracker equivalence in terms of statistical significance and practical differences. A fully-annotated dataset with per-frame annotations with several visual attributes is introduced. The diversity of its visual properties is maximized in a novel way by clustering a large number of videos according to their visual attributes. This makes it the most sophistically constructed and annotated dataset to date. A multi-platform evaluation system allowing easy integration of third-party trackers is presented as well. The proposed evaluation methodology was tested on the VOT2014 challenge on the new dataset and 38 trackers, making it the largest benchmark to date. Most of the tested trackers are indeed state-of-the-art since they outperform the standard baselines, resulting in a highly-challenging benchmark. An exhaustive analysis of the dataset from the perspective of tracking difficulty is carried out. To facilitate tracker comparison a new performance visualization technique is proposed.

Tracking by Natural Language Specification

“…The breadth and depth of single object tracking is covered by recent reviews [38,33] including an overview of the many diverse and dynamic factors to overcome conditions general to the scene: uneven illumination, shadow casting, reflection, as well as to the target: shape changes, and in relation to other objects: occlusion, similar close objects, clutter with the background, and the camera: fast motion and zooming. Diverse benchmarks like OTB [38], ALOV [33], and VOT [20], have accelerated the performance of trackers in general, and they have caused a convergence in tracking methods.…”

“…The common approach is to specify a target by means of a bounding box around the object and to track this target as it moves throughout the video [38,33,20]. The paradigm has proven to be effective and considerable progress has been achieved [17,37,34,3,11].…”

Visual Tracking using Invariant Feature Descriptor

“…Thus, it is inappropriate to calculate the sum of overlap region with respect to total number of frames. The tracking should be continued until fully processed the entire sequence to show the overall evaluation of the performance (Čehovin et al, 2016;Kristan et al, 2016). Both practices are implemented in our experimental study to show the comparative performance among trackers.…”

“…According to Kristan et al (2016), it is difficult to point out a 'best' visual tracker that is able to accommodate a large variety of surveillance dataset with diversified characteristics. Some applications require a highly accurate tracking over robustness, such as sport analytics that is used to locate the position of ball during tournament or calculate the player's acceleration and velocity.…”

Background Normalization and Texture Pattern-Based Video Segmentation for Visual Tracking

“…It is mainly used in the application such as robotics, visual surveillance (7), Human-to-Machine Interface (HMI) (8), video editing (9), motion control (10), (5), (11), (12), and activity recognition (6) to extract the target status. There are number of visual tracking systems to identify the targeted region from the frame of a given video.…”