Near-optimal learning of tree-structured distributions by Chow-Liu

Concentration Bounds for Discrete Distribution Estimation in KL Divergence

“…We provided concentration bounds for the KL divergence between the underlying distribution and the Laplace estimator. Our results show that the dependence on the error probability can be bounded as Õ( √ k log 5/2 (1/δ)/n), which improves on the previous bound of Õ(k log(1/δ)/n) recently obtained by [4]. We further established a lower bound of Ω( √ k/n) on the variance and the tail bound of the KL loss of the Laplace estimator, thus showing our results are nearly-optimal.…”

“…We refer the reader to Theorem 2 for the detailed statement, including leading constants and a bound holding for all regimes of n. We emphasize that, in contrast to the previous bound given in (5), the additional t δ term is here sublinear in k, and in particular negligible in front of the expectation term E[KL(p p1 )] for most values of δ. Viewed differently, our result improves on that of [4] for all δ ≥ exp(− Õ(k 1/3 )).…”

“…We note that [15] provides upper and lower bounds on the variance of E[KL(p p)] for the empirical estimator. To conclude this introduction, we mention a direct consequence of our result: plugging our main result, Theorem 1, into the proof of [4,Theorem 1.4] yields an improved sample complexity (to achieve…”

“…The best known for KL divergence is given by [4], who recently showed that for the Laplace estimator p1 , with probability at least 1 − δ,…”

On Approximating Total Variation Distance

“…Clearly, Bayes nets satisfy both conditions (where "efficient" means as usual polynomial in the number of parameters). Bhattacharyya, Gayen, Meel and Vinodchandran [BGMV20] extended this idea to develop polynomial-time algorithms for additively approximating the TV distance between two bounded in-degree Bayes nets using a polynomial number of samples from each.…”