Modern Subsampling Methods for Large-Scale Least Squares Regression

Abstract

Subsampling methods aim to select a subsample as a surrogate for the observed sample. As a powerful technique for large-scale data analysis, various subsampling methods are developed for more effective coefficient estimation and model prediction. This review presents some cutting-edge subsampling methods based on the large-scale least squares estimation. Two major families of subsampling methods are introduced, respectively, the randomized subsampling approach and the optimal subsampling approach.The former aims to develop a more effective data-dependent sampling probability, while the latter aims to select a deterministic subsample in accordance with certain optimality criteria. Real data examples are provided to compare these methods empirically, respecting both the estimation accuracy and the computing time.

Smoothing splines approximation using Hilbert curve basis selection

“…To alleviate the computation burden for smoothing splines, various basis selection methods have been developed. These methods are of similar nature to the subsampling methods, which are widely used in large-scale data analysis [Mahoney, 2011, Drineas et al, 2012, Ma et al, 2015b, Ma and Sun, 2015, Meng et al, 2017, Zhang et al, 2018b, Ai et al, 2021, Xie et al, 2019, Ma et al, 2020, Yu et al, 2020, Ai et al, 2020, Meng et al, 2020a, Li and Meng, 2021.…”