Mean Estimation and Regression Under Heavy-Tailed Distributions: A Survey

Lugosi, Gábor; Mendelson, Shahar

doi:10.1007/s10208-019-09427-x

Cited by 123 publications

(125 citation statements)

References 73 publications

Supporting

Mentioning

124

Contrasting

Order By: Relevance

“…To achieve robustness in both feature and response spaces, recent years have witnessed a rapid development of the "median-of-means" (MOM) principle, which dates back to [40] and [22], and a variety of MOM-based procedures for regression and classification in both low-an high-dimensional settings [12,13,26,27,33,35]. We refer to [34] for a recent survey. An interesting open problem is how to efficiently incorporate the MOM principle with nonconvex regularization or iteratively reweighted 1 -regularization so as to achieve high degree of robustness and variable selection consistency simultaneously.…”

Section: Related Literaturementioning

confidence: 99%

Iteratively reweighted ℓ1-penalized robust regression

Pan

Sun

Zhou

2021

Electron. J. Statist.

View full text Add to dashboard Cite

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moments, we show that iteratively reweighted 1 -penalized adaptive Huber regression estimator satisfies exponential deviation bounds and oracle properties, including the oracle convergence rate and variable selection consistency, under a weak beta-min condition. Computationally, we need as many as O(log s + log log d) iterations to reach such an oracle estimator, where s and d denote the sparsity and ambient dimension, respectively. Extension to a general class of robust loss functions is also considered. Numerical studies lend strong support to our methodology and theory.

show abstract

Section: Related Literaturementioning

confidence: 99%

Iteratively reweighted ℓ1-penalized robust regression

Pan

Sun

Zhou

2021

Electron. J. Statist.

View full text Add to dashboard Cite

show abstract

“…Robust statistics substantially broaden the capacity of traditional data models so that they can embrace real-world examples. For instance, a recent series of works such as Catoni (2012); Minsker (2015); Devroye et al (2016); Fan et al (2017; Lugosi and Mendelson (2019) study how to handle heavy-tailed data in point estimation and regression analysis. These works do not assume any parametric form of the data distribution but a bounded moment, and they show that the estimator based on median-of-means or robustified risk minimization exhibits sub-Gaussian behavior around the truth.…”

Section: Robust Statisticsmentioning

confidence: 99%

Modern Data Modeling: Cross-Fertilization of the Two Cultures

Fan¹,

Ma²,

Wang³

et al. 2021

Observational Studies

View full text Add to dashboard Cite

The past two decades have witnessed deep cross-fertilization between the two culturesstatistics (data/generative modeling) and machine learning (algorithmic modeling), which is in stark contrast to the scene pictured in Breiman's inspiring work. In light of this major confluence, we find it helpful to single out a few salient examples showcasing the impacts of one to the other, and the research progress out of them. We point out in the end that the current big data era especially requires joint efforts from both cultures in order to address some common challenges including decentralized data analysis, privacy, fairness, etc.

show abstract

“…where β > 0 is a parameter to be tuned and ϕ is non-decreasing and called influence function. The deviation performance of this estimator is much better than X. Catoni's idea has been broadly applied to many research problems, see for instance [1,15,5,6,7,11,12,17]. The finite variance assumption plays an important role in Catoni's analysis, but it rules out many interesting distributions such as Pareto law [10,16,4,8], which describes the distributions of wealth and social networks.…”

Section: Introductionmentioning

confidence: 99%

A generalized Catoni’s M-estimator under finite α-th moment assumption with α∈(1,2)

Chen

Jin

et al. 2021

Electron. J. Statist.

View full text Add to dashboard Cite

We generalize Catoni's M-estimator, put forward in [3] by Catoni under finite variance assumption, to the case in which distributions can have finite α-th moment with α ∈ (1, 2). Our approach, inspired by the Taylor-like expansion developed in [4], is via slightly modifying the influence function ϕ in [3]. A deviation bound is established for this generalized estimator, and coincides with that in [3] as α ↑ 2. Experiment shows that our M-estimator performs better than the empirical mean, the smaller the α is, the better the performance will be. As an application, we study an 1 regression considered by Zhang et al. [19], who assumed that samples have finite variance, under finite α-th moment assumption with α ∈ (1, 2).

show abstract

Mean Estimation and Regression Under Heavy-Tailed Distributions: A Survey

Cited by 123 publications

References 73 publications

Iteratively reweighted ℓ1-penalized robust regression

Iteratively reweighted ℓ1-penalized robust regression

Modern Data Modeling: Cross-Fertilization of the Two Cultures

A generalized Catoni’s M-estimator under finite α-th moment assumption with α∈(1,2)

Contact Info

Product

Resources

About