A Massive Data Framework for M-Estimators with Cubic-Rate

Abstract

The divide and conquer method is a common strategy for handling massive data. In this article, we study the divide and conquer method for cubic-rate estimators under the massive data framework. We develop a general theory for establishing the asymptotic distribution of the aggregated M-estimators using a simple average. Under certain condition on the growing rate of the number of subgroups, the resulting aggregated estimators are shown to have faster convergence rate and asymptotic normal distribution, which are more tractable in both computation and inference than the original M-estimators based on pooled data. Our theory applies to a wide class of M-estimators with cube root convergence rate, including the location estimator, maximum score estimator and value search estimator. Empirical performance via simulations also validate our theoretical findings.

Communication-Efficient Distributed Linear Discriminant Analysis for Binary Classification

“…In this way, we are able to break a large-scale computation problem into many small pieces, then solve them with divide-and-conquer procedures and communicate only certain summary statistics. In recent years, distributed statistical inference has received considerable attention, covering a wide range of topics including M-estimation (Chen and Xie, 2014;Rosenblatt and Nadler, 2016;Lee et al, 2017;Battey et al, 2018;Shi, Lu, and Song, 2018;Jordan et al, 2018;Banerjee, Durot, and Sen, 2019;Fan, Guo, and Wang, 2019), hypothesis test (Lalitha, Sarwate, and Javidi, 2014;Battey et al, 2018), confidence intervals (Jordan, Lee, and Yang, 2018;Chen, Liu, and Zhang, 2018;Dobriban and Sheng, 2018;Wang et al, 2019), principal component analysis (Garber, Shamir, and Srebro, 2017;, nonparametric regression (Zhang, Duchi, and Wainwright, 2015;Chang, Lin, and Zhou, 2017;Shang and Cheng, 2017;Han et al, 2018;Szabó and Van Zanten, 2019), Bayesian methods (Xu et al, 2014;Jordan et al, 2018), quantile regression (Volgushev, Chao, and Cheng, 2019;Chen, Liu, and Zhang, 2019), bootstrap inference (Kleiner et al, 2014;Han and Liu, 2016), and so on.…”

Score-matching representative approach for big data analysis with generalized linear models

“…Typically, its accuracy is not as good as the full data estimate. The divide-and-conquer idea has been applied to numerous statistical problems (Chen and Xie, 2014;Schifano et al, 2016;Zhao, Cheng and Liu, 2016;Lee et al, 2017;Battey et al, 2018;Shi, Lu and Song, 2018;Chen, Liu and Zhang, 2019). For example, Chen and Xie (2014) made an innovative attempt of majority voting on variable selection based on a divide-and-conquer framework that is similar in spirit to combining confidence distributions in meta-analysis (Singh, Xie and Strawderman, 2005;Xie, Singh and Strawderman, 2011).…”

Sampling‐based estimation for massive survival data with additive hazards model

“…Battey et al 2 investigated hypothesis testing and parameter estimation using the “divide and conquer” algorithm. Shi et al 3 studied the “divide and conquer” method for cubic‐rate estimators. Jordan et al 4 presented a communication‐efficient surrogate likelihood method for distributed statistical inference problems.…”