On Hoeffding’s inequalities

Abstract

In a celebrated work by Hoeffding [J. Amer. Statist. Assoc. 58 (1963) [13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30], several inequalities for tail probabilities of sums Mn = X1 + · · · + Xn of bounded independent random variables Xj were proved. These inequalities had a considerable impact on the development of probability and statistics, and remained unimproved until 1995 when Talagrand [Inst. HautesÉtudes Sci. Publ. Math. 81 (1995a) 73-205] inserted certain missing factors in the bounds of two theorems. By similar factors, a third theorem was refined by Pinelis [Progress in Probability 43 (1998) 257-314] and refined (and extended) by me. In this article, I introduce a new type of inequality. Namely, I show that P{Mn ≥ x} ≤ cP{Sn ≥ x}, where c is an absolute constant and Sn = ε1 + · · · + εn is a sum of independent identically distributed Bernoulli random variables (a random variable is called Bernoulli if it assumes at most two values). The inequality holds for those x ∈ R where the survival function x → P{Sn ≥ x} has a jump down. For the remaining x the inequality still holds provided that the function between the adjacent jump points is interpolated linearly or log-linearly. If it is necessary, to estimate P{Sn ≥ x} special bounds can be used for binomial probabilities. The results extend to martingales with bounded differences. It is apparent that Theorem 1.1 of this article is the most important. The inequalities have applications to measure concentration, leading to results of the type where, up to an absolute constant, the measure concentration is dominated by the concentration in a simplest appropriate model, such results will be considered elsewhere.

On Probability and Moment Inequalities for Supermartingales and Martingales

“…Generalizations of the Bernstein and Bennet-Hoeffding inequalities are obtained in [14], [15], [16], [17], and [18]. Probabilities of large deviations of Sn are studied in [19] under the condition max 1≤k≤n E|X k | t < ∞.…”

Sharp large deviation results for sums of independent random variables

“…Moreover, some tight lower bounds are also given, which were not considered by Talagrand [45], Bentkus [5] and Pinelis [36]. In particular, we improve Talagrand's inequality to an equality, which will imply simple large deviation expansions.…”

“…Other improvements on Hoeffding's bound can be found in Bentkus [5] and Pinelis [36]. Bentkus's inequality [5] is much better than (1.9) in the sense that it recovers a factor of order 1 x for all x 0 instead of the range 0 x σ Kb , and do not assume that ξ i 's have moments of order larger than 2; see also Pinelis [37] for a similar improvement on Bennett-Hoeffding's bound.…”

“…When the summands ξ i are bounded from above, results of such type have been obtained by Talagrand [45], Bentkus [5] and Pinelis [37]. Using the conjugate measure technique, Talagrand (cf.…”

“…The scope of this paper is to give several improvements on Bernstein's inequalities (1.3), (1.5) and Bennett's inequality (1.4) for sums of non-bounded random variables instead of sums of bounded (from above) random variables, which are considered in Talagrand [45], Bentkus [5] and Pinelis [36]. Moreover, some tight lower bounds are also given, which were not considered by Talagrand [45], Bentkus [5] and Pinelis [36].…”