On the concentration of the missing mass

Berend, Daniel; Kontorovich, Aryeh

doi:10.1214/ecp.v18-2359

Cited by 97 publications

(49 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where the last line follows directly from Theorem 1.1 in Schlemm (2016) (a result equivalent to Theorem 1.1 was also obtained in Berend and Kontorovich (2013)). Therefore, the standard Hanson-Wright inequality implies that with probability at least 1 − e −t we have…”

Section: Improving Hanson-wright Inequality In the Subgaussian Regimementioning

confidence: 66%

Uniform Hanson-Wright type concentration inequalities for unbounded entries via the entropy method

Klochkov¹,

Zhivotovskiy²

2020

Electron. J. Probab.

View full text Add to dashboard Cite

This paper is devoted to uniform versions of the Hanson-Wright inequality for a random vector X ∈ R n with independent subgaussian components. The core technique of the paper is based on the entropy method combined with truncations of both gradients of functions of interest and of the components of X itself. Our results recover, in particular, the classic uniform bound of Talagrand (1996) for Rademacher chaoses and the more recent uniform result of Adamczak (2015) which holds under certain rather strong assumptions on the distribution of X. We provide several applications of our techniques: we establish a version of the standard Hanson-Wright inequality, which is tighter in some regimes. Extending our results we show a version of the dimension-free matrix Bernstein inequality that holds for random matrices with a subexponential spectral norm. We apply the derived inequality to the problem of covariance estimation with missing observations and prove an almost optimal high probability version of the recent result of Lounici (2014). Finally, we show a uniform Hanson-Wright-type inequality in the Ising model under Dobrushin's condition. A closely related question was posed by Marton (2003).

show abstract

Section: Improving Hanson-wright Inequality In the Subgaussian Regimementioning

confidence: 66%

Uniform Hanson-Wright type concentration inequalities for unbounded entries via the entropy method

Klochkov¹,

Zhivotovskiy²

2020

Electron. J. Probab.

View full text Add to dashboard Cite

show abstract

“…thus indicating that 1 2g(µ) is a distribution-sensitive proxy variance for any [0, 1]-supported random variable with mean µ (see also Berend and Kontorovich, 2013, for a detailed proof of this result). If this is the optimal proxy variance for the Bernoulli distribution (see Theorem 2.1 and Theorem 3.1 of Buldygin and Moskvichova, 2013), it is clear from our result that it does not hold true for the Beta distribution.…”

Section: Introductionmentioning

confidence: 79%

“…The sub-Gaussian property Kozachenko, 1980, 2000;Pisier, 2016) and related concentration inequalities (Boucheron et al, 2013;Raginsky and Sason, 2013) have attracted a lot of attention in the last couple of decades due to their applications in various areas such as pure mathematics, physics, information theory and computer sciences. Recent interest focused on deriving the optimal proxy variance for discrete random variables like the Bernoulli distribution (Buldygin and Moskvichova, 2013;Kearns and Saul, 1998;Berend and Kontorovich, 2013) and the missing mass (McAllester and Schapire, 2000;McAllester and Ortiz, 2003;Berend and Kontorovich, 2013;Ben-Hamou et al, 2017). Our focus is instead on two continuous random variables, the Beta and Dirichlet distributions, for which the optimal proxy variance was not known to the best of our knowledge.…”

Section: Introductionmentioning

confidence: 99%

On the sub-Gaussianity of the Beta and Dirichlet distributions

Marchal¹,

Arbel²

2017

Electron. Commun. Probab.

View full text Add to dashboard Cite

We obtain the optimal proxy variance for the sub-Gaussianity of Beta distribution, thus proving upper bounds recently conjectured by Elder (2016). We provide different proof techniques for the symmetrical (around its mean) case and the non-symmetrical case. The technique in the latter case relies on studying the ordinary differential equation satisfied by the Beta moment-generating function known as the confluent hypergeometric function. As a consequence, we derive the optimal proxy variance for the Dirichlet distribution, which is apparently a novel result. We also provide a new proof of the optimal proxy variance for the Bernoulli distribution, and discuss in this context the proxy variance relation to log-Sobolev inequalities and transport inequalities.Comment: 13 pages, 2 figure

show abstract

“…Various properties of the Good-Turing estimator and several variations of it have been analyzed for distribution estimation and compression [9], [10], [11], [12], [13], [14], [15]. Several concentration results on missing mass estimation are also known [16], [17]. Despite all this work, the risk of the Good-Turing estimator and the minimax risk of missing mass estimation have still not been conclusively established.…”

Section: A Good-turing Estimator and Previous Resultsmentioning

confidence: 99%

Minimax risk for missing mass estimation

Rajaraman

Thangaraj

Suresh

2017

2017 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

Abstract-The problem of estimating the missing mass or total probability of unseen elements in a sequence of n random samples is considered under the squared error loss function. The worstcase risk of the popular Good-Turing estimator is shown to be between 0.6080/n and 0.6179/n. The minimax risk is shown to be lower bounded by 0.25/n. This appears to be the first such published result on minimax risk for estimation of missing mass, which has several practical and theoretical applications.

show abstract

On the concentration of the missing mass

Cited by 97 publications

References 6 publications

Uniform Hanson-Wright type concentration inequalities for unbounded entries via the entropy method

Uniform Hanson-Wright type concentration inequalities for unbounded entries via the entropy method

On the sub-Gaussianity of the Beta and Dirichlet distributions

Minimax risk for missing mass estimation

Contact Info

Product

Resources

About