Optimal Distributed Subsampling for Maximum Quasi-Likelihood Estimators With Massive Data

Yu, Jun; Wang, Haiying; Ai, Mingyao; Zhang, Huiming

doi:10.1080/01621459.2020.1773832

Cited by 75 publications

(37 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Pilot estimation. The idea of pilot is widely applied in subsampling procedure [7], [8], [19], [20], where the sampling probability is specified by a pilot estimation. A popular way for calculating pilot is uniform subsampling.…”

Section: Markov Subsampling Based On Huber Criterionmentioning

confidence: 99%

Markov subsampling based Huber Criterion

Gong¹,

Yan²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Subsampling is an important technique to tackle the computational challenges brought by big data. Many subsampling procedures fall within the framework of importance sampling, which assigns high sampling probabilities to the samples appearing to have big impacts. When the noise level is high, those sampling procedures tend to pick many outliers and thus often do not perform satisfactorily in practice. To tackle this issue, we design a new Markov subsampling strategy based on Huber criterion (HMS) to construct an informative subset from the noisy full data; the constructed subset then serves as a refined working data for efficient processing. HMS is built upon a Metropolis-Hasting procedure, where the inclusion probability of each sampling unit is determined using the Huber criterion to prevent over scoring the outliers. Under mild conditions, we show that the estimator based on the subsamples selected by HMS is statistically consistent with a sub-Gaussian deviation bound. The promising performance of HMS is demonstrated by extensive studies on large scale simulations and real data examples.

show abstract

Section: Markov Subsampling Based On Huber Criterionmentioning

confidence: 99%

Markov subsampling based Huber Criterion

Gong¹,

Yan²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, in the estimation problem of linear models, and Ma and Sun (2015) put forward a so-called algorithmic leveraging with a nonuniform sampling probability to draw a more informative subsample dataset. Some other recent developments in this trend include Wang et al (2019); Yao and Wang (2019); Yu et al (2020) and Ai et al (2021). However, the challenges associated with designing an efficient testing procedure for model checking are not yet well addressed.…”

Section: Introductionmentioning

confidence: 99%

Model Checking in Large-Scale Dataset via Structure-Adaptive-Sampling

Han¹,

Ma²,

Ren³

et al. 2023

STAT SINICA

View full text Add to dashboard Cite

Lack-of-fit testing is often essential in many applications of statistical/machine learning. Despite the availability of large-scale datasets, the challenges associated with model checking when some resource budgets are limited are not yet well addressed. In this paper, we propose a design-adaptive testing procedure to check a general model when only a limited number of data observations are available. We derive an optimal sampling strategy to select a small subset from a large pool of data, Structure-Adaptive-Sampling, with which the proposed test possesses the asymptotically best power. Numerical results on both synthetic and real-world data confirm the effectiveness of the proposed method.

show abstract

“…This method was named as optimal subsampling methods motivated from the A-optimality criterion (OSMAC), and was improved in Wang (2019b) by adopting unweighted target functions for subsamples and Poisson subsampling. In addition to logistic regression, OSMAC was investigated to include softmax regression (Yao and Wang, 2018), generalized linear models (Ai et al, 2019), quantile regression (Wang and Ma, 2020) and quasilikelihood (Yu et al, 2020). This article aims at introducing the optimal subsampling method and illustrates its practical implements in R (R Core Team, 2020) with the following real data examples.…”

Section: Introductionmentioning

confidence: 99%

A Review on Optimal Subsampling Methods for Massive Datasets

Yao¹,

Wang²

2021

Journal of Data Science

Self Cite

View full text Add to dashboard Cite

Subsampling is an effective way to deal with big data problems and many subsampling approaches have been proposed for different models, such as leverage sampling for linear regression models and local case control sampling for logistic regression models. In this article, we focus on optimal subsampling methods, which draw samples according to optimal subsampling probabilities formulated by minimizing some function of the asymptotic distribution. The optimal subsampling methods have been investigated to include logistic regression models, softmax regression models, generalized linear models, quantile regression models, and quasi-likelihood estimation. Real data examples are provided to show how optimal subsampling methods are applied.

show abstract

Optimal Distributed Subsampling for Maximum Quasi-Likelihood Estimators With Massive Data

Cited by 75 publications

References 22 publications

Markov subsampling based Huber Criterion

Markov subsampling based Huber Criterion

Model Checking in Large-Scale Dataset via Structure-Adaptive-Sampling

A Review on Optimal Subsampling Methods for Massive Datasets

Contact Info

Product

Resources

About