An Infinite-Dimensional Geometric Structure on the Space of all the Probability Measures Equivalent to a Given One

Pistone, Giovanni; Sempi, Carlo

doi:10.1214/aos/1176324311

Cited by 184 publications

(237 citation statements)

References 9 publications

Supporting

Mentioning

229

Contrasting

Unclassified

Order By: Relevance

“…The basic case of a finite state space has been extended by Amari and coworkers to the case of a parametric set of strictly positive probability densities on a generic sample space. Following a suggestion by Dawid in [7,8], a particular nonparametric version of that theory was developed in a series of papers [9][10][11][12][13][14][15][16][17][18][19], where the set P > of all strictly positive probability densities of a measure space is shown to be a Banach manifold (as it is defined in [20][21][22]) modeled on an Orlicz Banach space, see, e.g., [23, Chapter II].…”

Section: Introductionmentioning

confidence: 99%

“…This material is included for convenience only and this part should be skipped by any reader aware of any of the papers [9][10][11][12][13][14][15][16][17][18][19] quoted above. The following Section 3 is mostly based on the same references and it is intended to introduce that manifold structure and to give a first example of application to the study of Kullback-Liebler divergence.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Information Geometry Formalism for the Spatially Homogeneous Boltzmann Equation

Lods

Pistone

2015

Entropy

Self Cite

View full text Add to dashboard Cite

Information Geometry generalizes to infinite dimension by modeling the tangent space of the relevant manifold of probability densities with exponential Orlicz spaces. We review here several properties of the exponential manifold on a suitable set E of mutually absolutely continuous densities. We study in particular the fine properties of the Kullback-Liebler divergence in this context. We also show that this setting is well-suited for the study of the spatially homogeneous Boltzmann equation if E is a set of positive densities with finite relative entropy with respect to the Maxwell density. More precisely, we analyze the Boltzmann operator in the geometric setting from the point of its Maxwell's weak form as a composition of elementary operations in the exponential manifold, namely tensor product, conditioning, marginalization and we prove in a geometric way the basic facts, i.e., the H-theorem. We also illustrate the robustness of our method by discussing, besides the Kullback-Leibler divergence, also the property of Hyvärinen divergence. This requires us to generalize our approach to Orlicz-Sobolev spaces to include derivatives.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Information Geometry Formalism for the Spatially Homogeneous Boltzmann Equation

Lods

Pistone

2015

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…This class of divergences induces an invariant Riemannian metric given by the Fisher information matrix and a pair of invariant dual affine connections, the ±α-connections, which are not necessarily flat. See [13] for more delicate problems occurring in the function space.…”

Section: Introductionmentioning

confidence: 99%

Information geometry of divergence functions

Амари¹,

Cichocki²

2010

Bulletin of the Polish Academy of Sciences: Technical Sciences

107

152

View full text Add to dashboard Cite

Abstract. Measures of divergence between two points play a key role in many engineering problems. One such measure is a distance function, but there are many important measures which do not satisfy the properties of the distance. The Bregman divergence, KullbackLeibler divergence and f -divergence are such measures. In the present article, we study the differential-geometrical structure of a manifold induced by a divergence function. It consists of a Riemannian metric, and a pair of dually coupled affine connections, which are studied in information geometry. The class of Bregman divergences are characterized by a dually flat structure, which is originated from the Legendre duality. A dually flat space admits a generalized Pythagorean theorem. The class of f -divergences, defined on a manifold of probability distributions, is characterized by information monotonicity, and the Kullback-Leibler divergence belongs to the intersection of both classes. The f -divergence always gives the α-geometry, which consists of the Fisher information metric and a dual pair of ±α-connections. The α-divergence is a special class of f -divergences. This is unique, sitting at the intersection of the f -divergence and Bregman divergence classes in a manifold of positive measures. The geometry derived from the Tsallis q-entropy and related divergences are also addressed.

show abstract

“…The extension to more than one parameter is obvious. Note that in the mathematics literature also nonparametric models are considered [33,34].…”

Section: Final Remarksmentioning

confidence: 99%