Debiasing and Distributed Estimation for High-Dimensional Quantile Regression

Zhao, Weihua; Zhang, Fode; Lian, Heng

doi:10.1109/tnnls.2019.2933467

Cited by 48 publications

(20 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But this is not applicable when p is very large. Thus, for the high dimensional QR problem, Zhao et al (2014) and Zhao et al (2019) adopted an one-shot averaging method based on the debiased local estimates as that in (4). Accordingly, Chen et al ( 2020) proposed a communication-efficient multiround algorithm inspired by the approximate Newton method (Shamir et al, 2014).…”

Section: Non-smooth Loss Based Modelsmentioning

confidence: 99%

A review of distributed statistical inference

Gao

Liu

Wang

et al. 2021

Statistical Theory and Related Fields

View full text Add to dashboard Cite

The rapid emergence of massive datasets in various fields poses a serious challenge to traditional statistical methods. Meanwhile, it provides opportunities for researchers to develop novel algorithms. Inspired by the idea of divide-and-conquer, various distributed frameworks for statistical estimation and inference have been proposed. They were developed to deal with large-scale statistical optimization problems. This paper aims to provide a comprehensive review for related literature. It includes parametric models, nonparametric models, and other frequently used models. Their key ideas and theoretical properties are summarized. The trade-off between communication cost and estimate precision together with other concerns are discussed.

show abstract

Section: Non-smooth Loss Based Modelsmentioning

confidence: 99%

A review of distributed statistical inference

Gao

Liu

Wang

et al. 2021

Statistical Theory and Related Fields

View full text Add to dashboard Cite

show abstract

“…Our paper also makes a novel contribution to the literature on inference for high-dimensional quantile regression. Up until now, this literature is limited to different approaches for debiasing 1 -penalized estimates of the quantile regression vector when the response is homoscedastic (Zhao et al, 2019;Bradic and Kolar, 2017). Our approach differs in two aspects from this literature: First, our inference procedure allows for heteroscedastic responses.…”

Section: Prior and Related Workmentioning

confidence: 99%

“…This is conceptually very different from debiasing a regression vector and the resulting estimator has distinct theoretical properties. In particular, we can derive the limit distribution of the estimated conditional quantile function, whereas this is not possible when using the results from Zhao et al (2019); Bradic and Kolar (2017).…”

Section: Prior and Related Workmentioning

confidence: 99%

Debiased Inference on Heterogeneous Quantile Treatment Effects with Regression Rank-Scores

Giessing¹,

Wang²

2021

Preprint

View full text Add to dashboard Cite

Understanding treatment effect heterogeneity in observational studies is of great practical importance to many scientific fields because the same treatment may affect different individuals differently. Quantile regression provides a natural framework for modeling such heterogeneity. In this paper, we propose a new method for inference on heterogeneous quantile treatment effects that incorporates high-dimensional covariates. Our estimator combines a debiased 1 -penalized regression adjustment with a quantile-specific covariate balancing scheme. We present a comprehensive study of the theoretical properties of this estimator, including weak convergence of the heterogeneous quantile treatment effect process to the sum of two independent, centered Gaussian processes. We illustrate the finite-sample performance of our approach through Monte Carlo experiments and an empirical example, dealing with the differential effect of mothers' education on infant birth weights.

show abstract

“…Recently, distributed sparse linear regression has become increasingly popular; For instance, [17], [18] employ diffusion adaptation to maintain the sparsity of the learned linear models. Generally speaking, a central idea to guarantee sparsity in such models is to use ℓ 1 -regularization; For example, see [19], [20], [21]. The mentioned methods share all the model parameters in the network, making the communication cost be at least the order of data dimension.…”

Section: A Related Workmentioning

confidence: 99%

“…The mentioned methods share all the model parameters in the network, making the communication cost be at least the order of data dimension. Notably, [19] introduced a debiased LASSO model for a network-based setting that comes with some theoretical convergence guarantees. In [20], communication cost is the order of the data dimension, but there is an upper-bound for the steps needed for convergence.…”

Section: A Related Workmentioning

confidence: 99%

Distributed Sparse Feature Selection in Communication-Restricted Networks

Hanie¹,

Najafi²,

Motahari³

2021

Preprint

View full text Add to dashboard Cite

This paper aims to propose and theoretically analyze a new distributed scheme for sparse linear regression and feature selection. The primary goal is to learn the few causal features of a high-dimensional dataset based on noisy observations from an unknown sparse linear model. However, the presumed training set which includes 𝑛 data samples in R 𝑝 is already distributed over a large network with 𝑁 clients connected through extremely low-bandwidth links. Also, we consider the asymptotic configuration of 1 𝑁 𝑛 𝑝. In order to infer the causal dimensions from the whole dataset, we propose a simple, yet effective method for information sharing in the network. In this regard, we theoretically show that the true causal features can be reliably recovered with negligible bandwidth usage of 𝑂 (𝑁 log 𝑝) across the network. This yields a significantly lower communication cost in comparison with the trivial case of transmitting all the samples to a single node (centralized scenario), which requires 𝑂 (𝑛𝑝) transmissions. Even more sophisticated schemes such as ADMM still have a communication complexity of 𝑂 (𝑁 𝑝). Surprisingly, our sample complexity bound is proved to be the same (up to a constant factor) as the optimal centralized approach for a fixed performance measure in each node, while that of a naïve decentralized technique grows linearly with 𝑁. Theoretical guarantees in this paper are based on the recent analytic framework of debiased LASSO in [1], and are supported by several computer experiments performed on both synthetic and real-world datasets.

show abstract

Debiasing and Distributed Estimation for High-Dimensional Quantile Regression

Cited by 48 publications

References 24 publications

A review of distributed statistical inference

A review of distributed statistical inference

Debiased Inference on Heterogeneous Quantile Treatment Effects with Regression Rank-Scores

Distributed Sparse Feature Selection in Communication-Restricted Networks

Contact Info

Product

Resources

About