On The Equivalence of Projections in Relative (α- Entropy and Rényi Divergence

Karthik, P. N.; Sundaresan, Rajesh

doi:10.1109/ncc.2018.8599980

Cited by 4 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the following lemma we establish a connection between M (α) -family and E (α) -family. This is due to Karthik and Sundaresan [35,Th. 2], where it was proved for the discrete, canonical case.…”

Section: 72]mentioning

confidence: 95%

Projection Theorems and Estimating Equations for Power-Law Models

Gayen¹,

Kumar²

2019

Preprint

View full text Add to dashboard Cite

Projection theorems of divergence functionals reduce certain estimation problems under specific families of probability distributions to linear problems. In this paper, we study projection theorems concerning Kullback-Leibler, Rényi, density power, and logarithmic-density power divergences which are popular in robust inference. We first extend these projection theorems to the continuous case by directly solving the associated estimating equations. We then apply these ideas to solve certain estimation problems concerning Student and Cauchy distributions. Finally, we explore the projection theorems by a generalized notion of principle of sufficiency. In particular, we show that the statistics of the data that influence the projection theorems are also a minimal sufficient statistics with respect to this generalized notion.

show abstract

Section: 72]mentioning

confidence: 95%

Projection Theorems and Estimating Equations for Power-Law Models

Gayen¹,

Kumar²

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The α-exponential family (comprises the generalized Gaussians as subclass in the continuous case) was derived by minimizing Rényi divergence subject to linear constraints on the escort of the underlying distribution. Since these divergences and the families are closely related by the mapping p → p (α) [23], one would expect these two families be dual to each other with respect to FIM or the α-FIM. However, we show that this is not the case.…”

Section: The Generalized Crlb and Efficient Estimators We Derive The α-Version Of Thementioning

confidence: 99%

“…II]]. The measures p (α) and q (α) are called α-escort or α-scaled measures [23,36]. Observe from 1 that relative α-entropy is a monotone function of the Csiszár divergence, not between p and q, but their escorts p (α) and q (α) .…”

Section: Introductionmentioning

confidence: 99%

Cramér–Rao lower bounds arising from generalized Csiszár divergences

Kumar

Mishra

2020

Info. Geo.

View full text Add to dashboard Cite

We study the geometry of probability distributions with respect to a generalized family of Csiszár f -divergences. A member of this family is the relative α-entropy which is also a Rényi analog of relative entropy in information theory and known as logarithmic or projective power divergence in statistics. We apply Eguchi's theory to derive the Fisher information metric and the dual affine connections arising from these generalized divergence functions. This enables us to arrive at a more widely applicable version of the Cramér-Rao inequality, which provides a lower bound for the variance of an estimator for an escort of the underlying parametric probability distribution. We then extend the Amari-Nagaoka's dually flat structure of the exponential and mixer models to other distributions with respect to the aforementioned generalized metric. We show that these formulations lead us to find unbiased and efficient estimators for the escort model. Finally, we compare our work with prior results on generalized Cramér-Rao inequalities that were derived from non-information-geometric frameworks.

show abstract

“…While lemma 1 establishes the connection between M (α) and E α , lemma 2 establishes the connection between the estimation problem based on D α -divergence on E α and the estimation problem based on I α -divergence on M (α) . Lemma 1 is due to Karthik and Sundaresan [22,Th. 2], where only the reverse implication was proved though.…”

Section: Projection Equation For D α On α-Exponential Familymentioning

confidence: 99%

Projection Theorems of Divergences and Likelihood Maximization Methods

Gayen,

Kumar

2017

Preprint

View full text Add to dashboard Cite

Projection theorems of divergences enable us to find reverse projection of a divergence on a specific statistical model as a forward projection of the divergence on a different but rather "simpler" statistical model, which, in turn, results in solving a system of linear equations. Reverse projection of divergences are closely related to various estimation methods such as the maximum likelihood estimation or its variants in robust statistics. We consider projection theorems of three parametric families of divergences that are widely used in robust statistics, namely the Rényi divergences (or the Cressie-Reed power divergences), density power divergences, and the relative α-entropy (or the logarithmic density power divergences). We explore these projection theorems from the usual likelihood maximization approach and from the principle of sufficiency. In particular, we show the equivalence of solving the estimation problems by the projection theorems of the respective divergences and by directly solving the corresponding estimating equations. We also derive the projection theorem for the density power divergences.

show abstract

On The Equivalence of Projections in Relative (α- Entropy and Rényi Divergence

Cited by 4 publications

References 9 publications

Projection Theorems and Estimating Equations for Power-Law Models

Projection Theorems and Estimating Equations for Power-Law Models

Cramér–Rao lower bounds arising from generalized Csiszár divergences

Projection Theorems of Divergences and Likelihood Maximization Methods

Contact Info

Product

Resources

About