Approximating high-dimensional infinite-order $U$-statistics: Statistical and computational guarantees

Abstract

We study the problem of distributional approximations to highdimensional non-degenerate U -statistics with random kernels of diverging orders. Infinite-order U -statistics (IOUS) are a useful tool for constructing simultaneous prediction intervals that quantify the uncertainty of ensemble methods such as subbagging and random forests. A major obstacle in using the IOUS is their computational intractability when the sample size and/or order are large. In this article, we derive non-asymptotic Gaussian approximation error bounds for an incomplete version of the IOUS with a random kernel. We also study data-driven inferential methods for the incomplete IOUS via bootstraps and develop their statistical and computational guarantees.

High-dimensional central limit theorems by Stein’s method

“…Motivated by modern statistical applications in large-scale data, there has been a recent wave of interest in proving high-dimensional central limit theorems. Starting from the pioneering work by Chernozhukov, Chetverikov and Kato (2013), who established a Gaussian approximation for maxima of sums of centered independent random vectors, many articles have been devoted to the development of this subject: For example, see Chernozhukov, Chetverikov and Kato (2017a), for generalization to normal approximation on hyperrectangles and improvements of the error bound, Chen (2018), Chen and Kato (2019), Song, Chen and Kato (2019) for extensions to U -statistics, , Zhang and Cheng (2018), Zhang and Wu (2017) for sums of dependent random vectors and Belloni et al (2018) for a general survey and statistical applications. In particular, for W = n −1/2 n i=1 X i where {X 1 , .…”

Rates of convergence for random forests via generalized U-statistics

“…We note also that recent work by [48] considered a similar high-dimensional statistic and obtain the Berry-Esseen bound…”

“…(4.1) but to the power of 1/2 in our result in Theorem 3. The authors in [48] also argue that the term σ g 2 is lower bounded by O(s −2 ), ultimately implying that s is required to be at most n 1/4− , ∀ > 0, whereas we require only that s = O(n 1− ). It seems to be the multi -dimensionality alone rather than the high-dimensionality that is driving the suboptimal rates in Eq.…”

Central limit theorems for high dimensional dependent data

“…Since the seminal work Chernozhukov, Chetverikov and Kato (2013), there has been substantial progresses being made in several directions. For instances, generalization of the index set from the max-rectangles to hyper-rectangles with improved rates of convergence to normality can be found in Chernozhukov, Chetverikov and Kato (2017), Lopes, Lin and Mueller (2020), Deng and Zhang (2020), , Chernozhukov, Chetverikov and Koike (2020), Das and Lahiri (2020), Deng (2020), Koike (2020), Fang and Koike (2020), Lopes (2020), and Kuchibhotla and Rinaldo (2020); extension from linear sums to U -statistics with nonlinear kernels can be found in Chen (2018), Chen and Kato (2019), Song, Chen and Kato (2019), and Koike (2019); generalization to dependent random vectors over maxrecetangles can be found in Zhang and Wu (2017), Zhang andCheng (2018), and.…”