A new class of random vector entropy estimators and its applications in testing statistical hypotheses

Goria, M. N.; Leonenko, Nikolai; Mergel, Victor; Inverardi, Pier Luigi Novi

doi:10.1080/104852504200026815

Cited by 130 publications

(144 citation statements)

References 17 publications

Supporting

Mentioning

136

Contrasting

Order By: Relevance

“…A more general class of entropy estimators including the one above was shown to be asymptotically unbiased and consistent as N → ∞ in [8]. Using the entropy estimate we 1 We omit arrows from our network visualizations and implicitly assume the orientations to be directed away from the hip node.…”

Section: Learning the Graph Structurementioning

confidence: 99%

A Non-parametric Bayesian Network Prior of Human Pose

Lehrmann

Gehler

Nowozin

2013

2013 IEEE International Conference on Computer Vision

View full text Add to dashboard Cite

Having a sensible prior of human pose is a vital ingredient for many computer vision applications, including tracking and pose estimation. While the application of global non-parametric approaches and parametric models has led to some success, finding the right balance in terms of flexibility and tractability, as well as estimating model parameters from data has turned out to be challenging. In this work, we introduce a sparse Bayesian network model of human pose that is non-parametric with respect to the estimation of both its graph structure and its local distributions. We describe an efficient sampling scheme for our model and show its tractability for the computation of exact log-likelihoods. We empirically validate our approach on the Human 3.6M dataset and demonstrate superior performance to global models and parametric networks. We further illustrate our model's ability to represent and compose poses not present in the training set (compositionality) and describe a speed-accuracy trade-off that allows realtime scoring of poses.

show abstract

Section: Learning the Graph Structurementioning

confidence: 99%

A Non-parametric Bayesian Network Prior of Human Pose

Lehrmann

Gehler

Nowozin

2013

2013 IEEE International Conference on Computer Vision

View full text Add to dashboard Cite

show abstract

“…A correction of the bias has been derived in [31] in a different context. In the non-biased estimators of the (cross)-entropy the digamma function ψ(k) replaces the log(k) term:…”

Section: Non-parametric Estimation Of the Similarity Measurementioning

confidence: 99%

Sparse Multiscale Patches for Image Processing

Piro

Anthoine

Debreuve

et al. 2009

Emerging Trends in Visual Computing

View full text Add to dashboard Cite

Abstract. This paper presents a framework to define an objective measure of the similarity (or dissimilarity) between two images for image processing. The problem is twofold: 1) define a set of features that capture the information contained in the image relevant for the given task and 2) define a similarity measure in this feature space. In this paper, we propose a feature space as well as a statistical measure on this space. Our feature space is based on a global descriptor of the image in a multiscale transformed domain. After decomposition into a Laplacian pyramid, the coefficients are arranged in intrascale/interscale/interchannel patches which reflect the dependencies of neighboring coefficients in presence of specific structures or textures. At each scale, the probability density function (pdf) of these patches is used as a descriptor of the relevant information. Because of the sparsity of the multiscale transform, the most significant patches, called Sparse Multiscale Patches (SMP), describe efficiently these pdfs. We propose a statistical measure (the Kullback-Leibler divergence) based on the comparison of these probability density functions. Interestingly, this measure is estimated via the nonparametric, k-th nearest neighbor framework without explicitly building the pdfs. This framework is applied to a query-by-example image retrieval method. Experiments on two publicly available databases showed the potential of our SMP approach for this task. In particular, it performed comparably to a SIFT -based retrieval method and two versions of a fuzzy segmentation-based method (the UFM and CLUE methods), and it exhibited some robustness to different geometric and radiometric deformations of the images.

show abstract

“…In this paper we derive convergence rates for data-split versions of k-nearest neighbor (k-NN) estimators of Shannon and Rényi entropies proposed by Goria et.al. [12] and Leonenko et.al. [13] respectively.…”

Section: Introductionmentioning

confidence: 99%

“…Goria et.al. [12] and Leonenko et.al. [13] show that the estimators they propose are asymptotically unbiased and consistent.…”

Section: Introductionmentioning

confidence: 99%

“…Several estimators of Shannon entropy and Rényi entropy have been proposed for general multivariate densities f . These include consistent estimators based on entropic graphs [9,10], gap estimators [11], nearest neighbor distances [12,13,14,15], Edgeworth approximations [16], convex risk minimization [17] and kernel density estimates [18].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

k-nearest neighbor estimation of entropies with confidence

Sricharan

Raich

Hero

2011

2011 IEEE International Symposium on Information Theory Proceedings

View full text Add to dashboard Cite

Abstract-We analyze a k-nearest neighbor (k-NN) class of plug-in estimators for estimating Shannon entropy and Rényi entropy. Based on the statistical properties of k-NN balls, we derive explicit rates for the bias and variance of these plug-in estimators in terms of the sample size, the dimension of the samples and the underlying probability distribution. In addition, we establish a central limit theorem for the plug-in estimator that allows us to specify confidence intervals on the entropy functionals. As an application, we use our theory in anomaly detection problems to specify thresholds for achieving desired false alarm rates.

show abstract

A new class of random vector entropy estimators and its applications in testing statistical hypotheses

Cited by 130 publications

References 17 publications

A Non-parametric Bayesian Network Prior of Human Pose

A Non-parametric Bayesian Network Prior of Human Pose

Sparse Multiscale Patches for Image Processing

k-nearest neighbor estimation of entropies with confidence

Contact Info

Product

Resources

About