Information Theoretic Mean Shift Algorithm

Rao, S.S.; Liu, Weifeng; Prı́ncipe, José C.; Martins, Allan de Medeiros

doi:10.1109/mlsp.2006.275540

Cited by 17 publications

(15 citation statements)

References 6 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Cauchy-Schwarz divergence D CS is a symmetric measure, which obeys 0 ≤ D CS ≤ ∞ with the minimum obtained for p(x) = q(x) [38]. The measure inspired by the Cauchy-Schwarz inequality was derived as a part of the Information Theoretic Learning (ITL) framework [39] and its theoretical properties have been further investigated in [11].…”

Section: Cauchy-schwarz Divergencementioning

confidence: 99%

On Accuracy of PDF Divergence Estimators and Their Applicability to Representative Data Sampling

Budka

Gabrys

Musiał

2011

Entropy

View full text Add to dashboard Cite

Generalisation error estimation is an important issue in machine learning. Cross-validation traditionally used for this purpose requires building multiple models and repeating the whole procedure many times in order to produce reliable error estimates. It is however possible to accurately estimate the error using only a single model, if the training and test data are chosen appropriately. This paper investigates the possibility of using various probability density function divergence measures for the purpose of representative data sampling. As it turned out, the first difficulty one needs to deal with is estimation of the divergence itself. In contrast to other publications on this subject, the experimental results provided in this study show that in many cases it is not possible unless samples consisting of thousands of instances are used. Exhaustive experiments on the divergence guided representative data sampling have been performed using 26 publicly available benchmark datasets and 70 PDF divergence estimators, and their results have been analysed and discussed.

show abstract

Section: Cauchy-schwarz Divergencementioning

confidence: 99%

On Accuracy of PDF Divergence Estimators and Their Applicability to Representative Data Sampling

Budka

Gabrys

Musiał

2011

Entropy

View full text Add to dashboard Cite

show abstract

“…The hypothesis that the initial scenario distribution produced by the technique proposed in [78] is unimodal is confirmed by using the ITL Mean Shift algorithm [86], [87]. The Mean Shift is a nonparametric, mode-finding method that relies on a kernel-based construction of the search space surface.…”

Section: Modes Finding Using the Itl Mean Shift Algorithmmentioning

confidence: 99%

“…We use the same assumption for thermal generators, loads, and realized wind power availability as in [86]. The system consists of ten thermal generators with a mix of base, intermediate, and peak units.…”

Section: Overview Of Simulated Casesmentioning

confidence: 99%

See 1 more Smart Citation

Use of wind power forecasting in operational decisions.

Botterud¹,

Zhi²,

Wang³

et al. 2011

View full text Add to dashboard Cite

“…By playing directly with pdf of the data and estimating the entropy non-parametrically, ITL effectively goes beyond the second order statistics. The result is new cost functions that directly manipulate information, thus bringing in powerful techniques and applications in adaptive systems [13] and machine learning [14], [15]. …”

Section: Information Theoretic Learning (Itl)mentioning

confidence: 99%

Mean shift: An information theoretic perspective

Rao

Martins

Prı́ncipe

2009

Pattern Recognition Letters

Self Cite

View full text Add to dashboard Cite

Abstract-This paper develops a new understanding of mean shift algorithms from an information theoretic perspective. We show that the Gaussian Blurring Mean Shift (GBMS) directly minimizes the Renyi's quadratic entropy of the dataset and hence is unstable by definition. Further, its stable counterpart, the Gaussian Mean Shift (GMS), minimizes the Renyi's "cross" entropy where the local stationary solutions are modes of the dataset. By doing so, we aptly answer the question "What does mean shift algorithms optimize?", thus highlighting naturally the properties of these algorithms. A consequence of this new understanding is the superior performance of GMS over GBMS which we show in a wide variety of applications ranging from mode finding to clustering and image segmentation.Index Terms-Mean shift, information theoretic learning, Renyi's entropy.

show abstract

Information Theoretic Mean Shift Algorithm

Cited by 17 publications

References 6 publications

On Accuracy of PDF Divergence Estimators and Their Applicability to Representative Data Sampling

On Accuracy of PDF Divergence Estimators and Their Applicability to Representative Data Sampling

Use of wind power forecasting in operational decisions.

Mean shift: An information theoretic perspective

Contact Info

Product

Resources

About