A Subsampled Double Bootstrap for Massive Data

Abstract

The bootstrap is a popular and powerful method for assessing precision of estimators and inferential methods. However, for massive datasets which are increasingly prevalent, the bootstrap becomes prohibitively costly in computation and its feasibility is questionable even with modern parallel computing platforms. Recently Kleiner, Talwalkar, Sarkar, and Jordan (2014) proposed a method called BLB (Bag of Little Bootstraps) for massive data which is more computationally scalable with little sacrifice of statistical accuracy. Building on BLB and the idea of fast double bootstrap, we propose a new resampling method, the subsampled double bootstrap, for both independent data and time series data. We establish consistency of the subsampled double bootstrap under mild conditions for both independent and dependent cases. Methodologically, the subsampled double bootstrap is superior to BLB in terms of running time, more sample coverage and automatic implementation with less tuning parameters for a given time budget. Its advantage relative to BLB and bootstrap is also demonstrated in numerical simulations and a data illustration.

Bayesian Bootstraps for Massive Data

“…We have presented the BLBB and SDBB as two data-subsetting procedures to approximate the BB. The BLBB and SDBB are analogous to the BLB (Kleiner et al, 2014) and SDB (Sengupta et al, 2016). The proposed procedures have theoretical and computational properties that are comparable to those of their frequentist counterparts.…”

“…The SDBB is the Bayesian analogue to the subsampled double bootstrap for massive data proposed by Sengupta et al (2016), which also provides an approximation of ξ{π φ (·|X n )}. In Sengupta et al (2016), the authors claim that the SDB outperforms the BLB in some scenarios with limited time budget, especially when it is only possible to run s < n/b little bootstraps. Therefore, we would expect the same phenomenon to occur with the BLBB and SDBB.…”

“…However, the BLBB could be outperformed by the SDBB because the little Bayesian bootstraps only consider a unique partition of the dataset and, if the computational budget is limited, only a fraction of the dataset contributes to the analysis. We refer to this fraction as sample coverage, a term we borrow from Sengupta et al (2016).…”

“…The technical conditions assumed for T and P 0 in Theorem 2 are similar to those assumed for the BLBB. Theorem 2 is the counterpart of Theorem 1 in Sengupta et al (2016). Figure 1 describes the Monte Carlo algorithm for running the SDBB.…”

Topics in statistical inference for massive data and high-dimensional data

“…Among the techniques developed for analyzing massive datasets, there are two categories that are most important. One is the "split-and-conquer" method (Zhang, Duchi and Wainwright, 2013;Chen and Xie, 2014;Battey, et al, 2015), and the other one is the resampling-based methods (Kleiner, et al, 2014;Sengupta, Volgushev and Shao, 2016). In this paper, we consider a general class of symmetric statistics (Lai and Wang, 1993;Jing and Wang, 2010) that encompasses many commonly used statistics, for example, the U and L-statistics.…”