Projection theorems and estimating equations for power-law models

Gayen, Atin; Kumar, M. Ashok

doi:10.1016/j.jmva.2021.104734

Cited by 8 publications

(3 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Statistical inference and machine learning involving generalized exponential families is the subject of a recent line of work, for e.g. [ 50 , 51 ]. We expect that -duality and logarithmic divergences will be useful in this endeavor.…”

Section: Discussion and Future Directionsmentioning

confidence: 99%

Conformal mirror descent with logarithmic divergences

Kainth

Wong²,

Rudzicz³

2022

Info. Geo.

View full text Add to dashboard Cite

The logarithmic divergence is an extension of the Bregman divergence motivated by optimal transport and a generalized convex duality, and satisfies many remarkable properties. Using the geometry induced by the logarithmic divergence, we introduce a generalization of continuous time mirror descent that we term the conformal mirror descent. We derive its dynamics under a generalized mirror map, and show that it is a time change of a corresponding Hessian gradient flow. We also prove convergence results in continuous time. We apply the conformal mirror descent to online estimation of a generalized exponential family, and construct a family of gradient flows on the unit simplex via the Dirichlet optimal transport problem.

show abstract

Section: Discussion and Future Directionsmentioning

confidence: 99%

Conformal mirror descent with logarithmic divergences

Kainth

Wong²,

Rudzicz³

2022

Info. Geo.

View full text Add to dashboard Cite

show abstract

“…. dλpxq) in the divergences ( 61) to (73) λ´probability-density functions. First, in the "λ´probability-density functions" context of Remark 1 one has for general X the statistical functionals S λpd…”

Section: Sxpqqmentioning

confidence: 99%

“…As a third statistical incentive, let us mention that with the help of φ´divergence minimization one can build generalizations of exponential families with pregiven sufficient statistics (see e.g. Pelletier [157], Gayen & Kumar [73]). In the special case of Kullback-Leiber divergence (i.e., the divergence generator φ is taken to be φ 1 ptq " φ KL ptq " t logptq `1 ´t) one ends up with classical exponential families.…”

mentioning

confidence: 99%

A Unifying Framework for Some Directed Distances in Statistics

Broniatowski¹,

Stummer²

2022

Preprint

View full text Add to dashboard Cite

Density-based directed distances -particularly known as divergences -between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine learning. Prominent examples are the Kullback-Leibler information distance (relative entropy) which e.g. is closely connected to the omnipresent maximum likelihood estimation method, and Pearson's χ 2 ´distance which e.g. is used for the celebrated chisquare goodness-of-fit test. Another line of statistical inference is built upon distribution-function-based divergences such as e.g. the prominent (weighted versions of) Cramer-von Mises test statistics respectively Anderson-Darling test statistics which are frequently applied for goodness-of-fit investigations; some more recent methods deal with (other kinds of) cumulative paired divergences and closely related concepts. In this paper, we provide a general framework which covers in particular both the abovementioned density-based and distribution-function-based divergence approaches; the dissimilarity of quantiles respectively of other statistical functionals will be included as well. From this framework, we structurally extract numerous classical and also state-of-the-art (including new) procedures. Furthermore, we deduce new concepts of dependence between random variables, as alternatives to the celebrated mutual information. Some variational representations are discussed, too.

show abstract

Computing Statistical Divergences with Sigma Points

Nielsen

Nock

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Projection theorems and estimating equations for power-law models

Cited by 8 publications

References 46 publications

Conformal mirror descent with logarithmic divergences

Conformal mirror descent with logarithmic divergences

A Unifying Framework for Some Directed Distances in Statistics

Computing Statistical Divergences with Sigma Points

Contact Info

Product

Resources

About