On weakly bounded empirical processes

Abstract

Let F be a class of functions on a probability space (Ω, µ) and let X 1 , ..., X k be independent random variables distributed according to µ. We establish high probability tail estimates of the form sup f ∈F |{i : |f (X i )| ≥ t} using a natural parameter associated with F . We use this result to analyze weakly bounded empirical processes indexed by F and processes of the formWe also present some geometric applications of this approach, based on properties of the random operatorare sampled according to an isotropic, log-concave measure on R n .

Aggregation via empirical risk minimization

“…Nevertheless, it is possible to obtain similar results under milder assumptions on Y and F, using the methods developed in [9,18,19] for handling empirical processes indexed by powers of unbounded function classes that satisfy suitable tail assumptions. Since the analysis of the unbounded case is technically much harder and would shift the emphasis of this article away from the main ideas we wish to present, we will only consider the uniformly bounded case.…”

Introduction to the non-asymptotic analysis of random matrices

“…A version of Theorem 5.65 was proved in [54] for the row-independent model; an extension from sub-gaussian to sub-exponential distributions is given in [3]. A general framework of stochastic processes with sub-exponential tails is discussed in [52]. For the column-independent model, Theorem 5.65 seems to be new.…”

Empirical Processes with a Bounded Ψ 1 Diameter

“…Currently, the best estimate on (6.3) in the range c(β)n 1+β ≤ N ≤ exp( √ n) for any β > 0 is c 1 (β) (n log n)/N . This is a corollary of Theorem A, and the suboptimal estimate from [25], that γ 2 (S n−1 , ψ 2 ) √ n log n for µ that is supported in c 2 √ nB n 2 (the so-called small diameter case). Note that the small diameter assumption can be made without loss of generality as long as N ≤ exp( √ n) thanks to the result of Paouris [30] which states that for N ≤ exp( √ n), E max i≤N X i 2 √ n. Hence, for those values of N , one may assume that µ is supported in c 2 √ nB n 2 , implying that if c(β)n 1+β ≤ N ≤ exp( √ n) then Theorem A improves Theorem 6.4 and gives the best known estimate on (6.3).…”

“…Let us mention that the estimate for p = 1 (the weakest of all the estimates for 1 ≤ p ≤ 2) was proved in [25] using a simpler chaining argument.…”

“…Thus, even with a sharp decomposition theorem at our disposal, a contraction based estimate on the empirical process indexed by F 2 leads to a superfluous log N factor. Despite that, this type of a decomposition argument is strong enough for many applications (see, for example, [9,15,25], and most notably, in [1]), because in those cases the all the required information is when d ψ 1 √ N is proportional to the complexity parameter of the class, rather than for larger values of N .…”