On Delta-Method of Moments and Probabilistic Sums

Cichoń, Jacek; Gołębiewski, Zbigniew; Kardas, Marcin; Klonowski, Marek

doi:10.1137/1.9781611973037.11

Cited by 7 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the literature of combinatorics, the sum n j=0 a j,n B j,n (x) is called the Bernoulli sum, and various approaches have been proposed to evaluate its asymptotics[58],[59],[60].…”

mentioning

confidence: 99%

Maximum Likelihood Estimation of Functionals of Discrete Distributions

Jiao

Venkat

Han

et al. 2017

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We consider the problem of estimating functionals of discrete distributions, and focus on tight nonasymptotic analysis of the worst case squared error risk of widely used estimators. We apply concentration inequalities to analyze the random fluctuation of these estimators around their expectations, and the theory of approximation using positive linear operators to analyze the deviation of their expectations from the true functional, namely their \emph{bias}. We characterize the worst case squared error risk incurred by the Maximum Likelihood Estimator (MLE) in estimating the Shannon entropy $H(P) = \sum_{i = 1}^S -p_i \ln p_i$, and $F_\alpha(P) = \sum_{i = 1}^S p_i^\alpha,\alpha>0$, up to multiplicative constants, for any alphabet size $S\leq \infty$ and sample size $n$ for which the risk may vanish. As a corollary, for Shannon entropy estimation, we show that it is necessary and sufficient to have $n \gg S$ observations for the MLE to be consistent. In addition, we establish that it is necessary and sufficient to consider $n \gg S^{1/\alpha}$ samples for the MLE to consistently estimate $F_\alpha(P), 0<\alpha<1$. The minimax rate-optimal estimators for both problems require $S/\ln S$ and $S^{1/\alpha}/\ln S$ samples, which implies that the MLE has a strictly sub-optimal sample complexity. When $1<\alpha<3/2$, we show that the worst-case squared error rate of convergence for the MLE is $n^{-2(\alpha-1)}$ for infinite alphabet size, while the minimax squared error rate is $(n\ln n)^{-2(\alpha-1)}$. When $\alpha\geq 3/2$, the MLE achieves the minimax optimal rate $n^{-1}$ regardless of the alphabet size. As an application of the general theory, we analyze the Dirichlet prior smoothing techniques for Shannon entropy estimation. We show that no matter how we tune the parameters in the Dirichlet prior, this technique cannot achieve the minimax rates in entropy estimation.Comment: 27 pages, 1 figure, published in IEEE Transactions on Information Theor

show abstract

“…In the literature of combinatorics, the sum n j=0 a j,n B j,n (x) is called the Bernoulli sum, and various approaches have been proposed to evaluate its asymptotics[58],[59],[60].…”

mentioning

confidence: 99%

Maximum Likelihood Estimation of Functionals of Discrete Distributions

Jiao

Venkat

Han

et al. 2017

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…We remark that asymptotic expansions for the entropy of binomial and negative binomial distributions for large mean (in terms of the difference between the entropy and the Gaussian entropy estimate) has been derived in [JS99] and [CGKK13] (and, among other results, in [DV98] and [Kne98]).…”

Section: Introductionmentioning

confidence: 84%

Expressions for the Entropy of Binomial-Type Distributions

Cheraghchi

2018

2018 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

We develop a general method for computing logarithmic and log-gamma expectations of distributions. As a result, we derive series expansions and integral representations of the entropy for several fundamental distributions, including the Poisson, binomial, beta-binomial, negative binomial, and hypergeometric distributions. Our results also establish connections between the entropy functions and to the Riemann zeta function and its generalizations. * A preliminary version of this work appears in Proceedings of the 2018 IEEE International Symposium on Information Theory (ISIT). †

show abstract

“…This allows us to analytically compute their entanglement entropy using approximations to the entropy of the hypergeometric distribution [59],…”

Section: Average Entanglement Entropy Of Dicke Basismentioning

confidence: 99%

“…For N A ∝ N = 2j the entropy of the hypergeometric distribution (and the EE of Dicke states) is approximated by [59]…”

Section: A Entanglement Of Dicke States and The Lmg Modelmentioning

confidence: 99%

Eigenstate entanglement in integrable collective spin models

Kumari

Alhambra

2022

Quantum

View full text Add to dashboard Cite

The average entanglement entropy (EE) of the energy eigenstates in non-vanishing partitions has been recently proposed as a diagnostic of integrability in quantum many-body systems. For it to be a faithful characterization of quantum integrability, it should distinguish quantum systems with a well-defined classical limit in the same way as the unequivocal classical integrability criteria. We examine the proposed diagnostic in the class of collective spin models characterized by permutation symmetry in the spins. The well-known Lipkin-Meshov-Glick (LMG) model is a paradigmatic integrable system in this class with a well-defined classical limit. Thus, this model is an excellent testbed for examining quantum integrability diagnostics. First, we calculate analytically the average EE of the Dicke basis{|j,m⟩}m=−jjin any non-vanishing bipartition, and show that in the thermodynamic limit, it converges to1/2of the maximal EE in the corresponding bipartition. Using finite-size scaling, we numerically demonstrate that the aforementioned average EE in the thermodynamic limit is universal for all parameter values of the LMG model. Our analysis illustrates how a value of the average EE far away from the maximal in the thermodynamic limit could be a signature of integrability.

show abstract

On Delta-Method of Moments and Probabilistic Sums

Cited by 7 publications

References 12 publications

Maximum Likelihood Estimation of Functionals of Discrete Distributions

Maximum Likelihood Estimation of Functionals of Discrete Distributions

Expressions for the Entropy of Binomial-Type Distributions

Eigenstate entanglement in integrable collective spin models

Contact Info

Product

Resources

About