On Divergences and Informations in Statistics and Information Theory

Liese, Friedrich; Vajda, Igor

doi:10.1109/tit.2006.881731

Cited by 503 publications

(506 citation statements)

References 33 publications

Supporting

Mentioning

495

Contrasting

Unclassified

Order By: Relevance

“…(The relative modular operator ∆(ρ 1 /ρ 2 ) was born in the context of von Neumann algebras and the paper of Araki [2] had a big influence even in the matrix case.) The quasientropy is the quantum generalization of the f -divergence of Csiszár used in classical information theory (and statistics) [3,20]. Therefore the quantum f -divergence could be another terminology as in [10].…”

Section: Quasi-entropymentioning

confidence: 99%

From Quasi-Entropy to Various Quantum Information Quantities

Hiai¹,

Petz²

2012

Publ. Res. Inst. Math. Sci.

View full text Add to dashboard Cite

The subject is the applications of the use of quasi-entropy in finite dimensional spaces to many important quantities in quantum information. Operator monotone functions and relative modular operators are used. The origin is the relative entropy, and the f -divergence, monotone metrics, covariance and the χ 2 -divergence are the most important particular cases. The extension of monotone metrics to those with two parameters is a new concept. Monotone metrics are also characterized by their joint convexity property.2000 Mathematics Subject Classification. Primary 81P45; Secondary 54C70.

show abstract

Section: Quasi-entropymentioning

confidence: 99%

From Quasi-Entropy to Various Quantum Information Quantities

Hiai¹,

Petz²

2012

Publ. Res. Inst. Math. Sci.

View full text Add to dashboard Cite

show abstract

“…where the arrows ↑ and ↓ denote the limit from the left and the limit from the right, respectively (see [30] for notations).…”

Section: Skew Burbea-rao Divergencesmentioning

confidence: 99%

“…31 is the right derivative (see Theorem 1 of [30] that gives a generalized Taylor expansion of convex functions) of the map…”

Section: Skew Burbea-rao Divergencesmentioning

confidence: 99%

The Burbea-Rao and Bhattacharyya centroids

Nielsen¹

2010

SciVee

140

View full text Add to dashboard Cite

Abstract-We study the centroid with respect to the class of information-theoretic Burbea-Rao divergences that generalize the celebrated Jensen-Shannon divergence by measuring the nonnegative Jensen difference induced by a strictly convex and differentiable function. Although those Burbea-Rao divergences are symmetric by construction, they are not metric since they fail to satisfy the triangle inequality. We first explain how a particular symmetrization of Bregman divergences called JensenBregman distances yields exactly those Burbea-Rao divergences. We then proceed by defining skew Burbea-Rao divergences, and show that skew Burbea-Rao divergences amount in limit cases to compute Bregman divergences. We then prove that Burbea-Rao centroids are unique, and can be arbitrarily finely approximated by a generic iterative concave-convex optimization algorithm with guaranteed convergence property. In the second part of the paper, we consider the Bhattacharyya distance that is commonly used to measure overlapping degree of probability distributions. We show that Bhattacharyya distances on members of the same statistical exponential family amount to calculate a Burbea-Rao divergence in disguise. Thus we get an efficient algorithm for computing the Bhattacharyya centroid of a set of parametric distributions belonging to the same exponential families, improving over former specialized methods found in the literature that were limited to univariate or "diagonal" multivariate Gaussians. To illustrate the performance of our Bhattacharyya/Burbea-Rao centroid algorithm, we present experimental performance results for k-means and hierarchical clustering methods of Gaussian mixture models.

show abstract

“…In order to remove this drawback, some proposals have been made in literature. Among them, we recall the minimum density power divergence method introduced by Basu et al (1998), and a minimum divergence method based on duality arguments, independently proposed by Liese and Vajda (2006) and Broniatowsi and Keziou (2009). The results obtained in the present paper follow this line of research.…”

Section: Introductionmentioning

confidence: 99%

Decomposable pseudodistances and applications in statistical estimation

Broniatowski

Toma

Vajda

2012

Journal of Statistical Planning and Inference

View full text Add to dashboard Cite

The aim of this paper is to introduce new statistical criterions for estimation, suitable for inference in models with common continuous support. This proposal is in the direct line of a renewed interest for divergence based inference tools imbedding the most classical ones, such as maximum likelihood, Chi-square or Kullback Leibler. General pseudodistances with decomposable structure are considered, they allowing to define minimum pseudodistance estimators, without using nonparametric density estimators. A special class of pseudodistances indexed by α > 0, leading for α ↓ 0 to the Kulback Leibler divergence, is presented in detail. Corresponding estimation criteria are developed and asymptotic properties are studied. The estimation method is then extended to regression models. Finally, some examples based on Monte Carlo simulations are discussed.

show abstract

On Divergences and Informations in Statistics and Information Theory

Cited by 503 publications

References 33 publications

From Quasi-Entropy to Various Quantum Information Quantities

From Quasi-Entropy to Various Quantum Information Quantities

The Burbea-Rao and Bhattacharyya centroids

Decomposable pseudodistances and applications in statistical estimation

Contact Info

Product

Resources

About