Adaptive Online Performance Evaluation of Video Trackers

SanMiguel, Juan C.; Cavallaro, Andrea; Martinez, José M.

doi:10.1109/tip.2011.2182520

Cited by 33 publications

(42 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To sidestep this issue, Wu et al [36] have proposed a protocol for stochastic tracker evaluation on a selected dataset that does not require ground truth labels. A similar approach was adapted by [37] to evaluate tracking algorithms on long sequences. Datasets with various visual phenomena equally represented are not usually used.…”

Section: Datasetsmentioning

confidence: 99%

A Novel Performance Evaluation Methodology for Single-Target Trackers

Kristan

Matas

Leonardis

et al. 2016

IEEE Trans. Pattern Anal. Mach. Intell.

582

440

View full text Add to dashboard Cite

Abstract-This paper addresses the problem of single-target tracker performance evaluation. We consider the performance measures, the dataset and the evaluation system to be the most important components of tracker evaluation and propose requirements for each of them. The requirements are the basis of a new evaluation methodology that aims at a simple and easily interpretable tracker comparison. The ranking-based methodology addresses tracker equivalence in terms of statistical significance and practical differences. A fully-annotated dataset with per-frame annotations with several visual attributes is introduced. The diversity of its visual properties is maximized in a novel way by clustering a large number of videos according to their visual attributes. This makes it the most sophistically constructed and annotated dataset to date. A multi-platform evaluation system allowing easy integration of third-party trackers is presented as well. The proposed evaluation methodology was tested on the VOT2014 challenge on the new dataset and 38 trackers, making it the largest benchmark to date. Most of the tested trackers are indeed state-of-the-art since they outperform the standard baselines, resulting in a highly-challenging benchmark. An exhaustive analysis of the dataset from the perspective of tracking difficulty is carried out. To facilitate tracker comparison a new performance visualization technique is proposed.

show abstract

Section: Datasetsmentioning

confidence: 99%

A Novel Performance Evaluation Methodology for Single-Target Trackers

Kristan

Matas

Leonardis

et al. 2016

IEEE Trans. Pattern Anal. Mach. Intell.

582

440

View full text Add to dashboard Cite

show abstract

“…This model is continuously updated after each processed frame and used to estimate the objects' positions in the next frame. Measuring the accuracy of a tracking algorithm can be performed by comparing the trackers output with the ground-truth [47]. Tracking is considered to be successful if an object's location and description provided by the tracker matches the ground-truth data.…”

Section: Utility Evaluation By Object Trackingmentioning

confidence: 99%

Privacy protection vs. utility in visual data

Erdélyi

Winkler

Rinner

2017

Multimed Tools Appl

View full text Add to dashboard Cite

Ubiquitous and networked sensors impose a huge challenge for privacy protection which has become an emerging problem of modern society. Protecting the privacy of visual data is particularly important due to the omnipresence of cameras, and various protection mechanisms for captured images and videos have been proposed. This paper introduces an objective evaluation framework in order to assess such protection methods. Visual privacy protection is typically realised by obfuscating sensitive image regions which often results in some loss of utility. Our evaluation framework assesses the achieved privacy protection and utility by comparing the performance of standard computer vision tasks, such as object recognition, detection and tracking on protected and unprotected visual data. The proposed framework extends the traditional frame-by-frame evaluation approach by introducing two new approaches based on aggregated and fused frames. We demonstrate our framework on eight differently protected video-sets and measure the trade-off between the improved privacy protection due to obfuscating captured image data and the degraded utility of the visual data. Results provided by our objective evaluation method are compared with an available state-of-the-art subjective study of these eight protection techniques.

show abstract

“…Standalone evaluation approaches for multi-hypothesis tracking have proven their superior performance as compared to singlehypothesis ones [2][3] [4]. Multi-hypothesis approaches require estimating the posterior distribution of the tracked target and cannot be applied directly to evaluate deterministic (single-hypothesis) tracking.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we present a standalone performance evaluation approach that adapts a multi-hypothesis strategy [4] to deterministic tracking by converting its single-hypothesis localization process into a distribution of the target state which emulates a multi-hypothesis Fig. 1.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Standalone evaluation of deterministic video tracking

SanMiguel

Cavallaro

Martinez

2012

2012 19th IEEE International Conference on Image Processing

View full text Add to dashboard Cite

Esta es la versión de autor de la comunicación de congreso publicada en: This is an author produced version of a paper published in: ABSTRACTWe present an approach for performance evaluation of deterministic video trackers without ground-truth data. The proposed approach detects if a tracker is correctly operating over time using two main steps. First, it transforms the output of the localization step into a distribution of the target state, which emulates a multi-hypothesis tracker. Then, the uncertainty of such distribution is estimated to determine the time instants when the tracker is stable. A time-reversed analysis is used to identify tracker recovery after unsuccessful operation. The proposed approach is demonstrated on the well-known MeanShift tracker. The results over a heterogeneous dataset show that the proposed approach outperforms the related state-of-the-art methods in presence of tracking challenges such as occlusions, illumination and scale changes, and clutter.

show abstract

Adaptive Online Performance Evaluation of Video Trackers

Cited by 33 publications

References 28 publications

A Novel Performance Evaluation Methodology for Single-Target Trackers

A Novel Performance Evaluation Methodology for Single-Target Trackers

Privacy protection vs. utility in visual data

Standalone evaluation of deterministic video tracking

Contact Info

Product

Resources

About