Gaussian approximation of suprema of empirical processes

Abstract

This paper develops a new direct approach to approximating suprema of general empirical processes by a sequence of suprema of Gaussian processes, without taking the route of approximating whole empirical processes in the sup-norm. We prove an abstract approximation theorem applicable to a wide variety of statistical problems, such as construction of uniform confidence bands for functions. Notably, the bound in the main approximation theorem is nonasymptotic and the theorem does not require uniform boundedness of the class of functions. The proof of the approximation theorem builds on a new coupling inequality for maxima of sums of random vectors, the proof of which depends on an e↵ective use of Stein's method for normal approximation, and some new empirical process techniques. We study applications of this approximation theorem to local and series empirical processes arising in nonparametric estimation via kernel and series methods, where the classes of functions change with the sample size and are non-Donsker. Importantly, our new technique is able to prove the Gaussian approximation for the supremum type statistics under weak regularity conditions, especially concerning the bandwidth and the number of series functions, in those examples.

Central limit theorems and bootstrap in high dimensions

“…Various bounds on ̺ and on closely related quantities were derived in [12,13,15,42,20,41,34,30,17,23,32,31,19,18] but in our discussion, we only focus on the results that are particular relevant for comparisons with our results. In addition, for clarity of the introduction, we assume below that components of X i 's are uniformly bounded by the envelope constant B n = B n (d), i.e.…”

On Gaussian comparison inequality and its application to spectral analysis of large random matrices

“…Hence, a heuristic sketch is useful. A major ingredient stems from a Gaussian comparison inequality recently developed by Chernozhukov, Chetverikov and Kato [14].…”

“…1 To study the limiting behavior of Q max in high dimensions, a major insight is to build the connection between the analysis of the maximum eigenvalue and recent developments in extreme value theory. In particular, by viewing the maximum eigenvalue as the extreme value of a specific infinite-state stochastic process, the Gaussian comparison inequality recently developed in Chernozhukov, Chetverikov and Kato [14] can be used. New empirical process bounds are established to ensure the validity of the inference procedure.…”

Inference in Nonparametric Series Estimation With Specification Searches for the Number of Series Terms

“…The rate restriction is mild because it only requires K/n 1−2/q → 0, up to log n terms, in case (a) and K/n → 0, up to log n terms, in case (b) when ζ K √ K for splines and wavelet series. Theorem 3.1 builds upon a coupling inequality for maxima of sums of random vectors in Chernozhukov et al (2014a) combined with the anticoncentration inequality in Chernozhukov et al (2014b). Remark 3.1 (Undersmoothing assumption).…”

“…For uniform inference, we require similar but slightly stronger conditions compared to Assumption 3.2. We also impose mild rate restrictions on ζ L 1 ,ζ L 2 and max K∈K n ζ K similar to Chernozhukov et al (2014a) and Belloni et al (2015). THEOREM 4.1.…”