Agnostic Estimation of Mean and Covariance

Lai, Kevin A.; Rao, Anup; Vempala, Santosh

doi:10.48550/arxiv.1604.06968

Cited by 2 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is worth noting that in contrast to prior work the sample complexity, has a logarithmic dependence on the ambient dimension d, allowing for high-dimensional scalings where d ≫ n, provided that the sparsity s 2 ≪ n. As in the work of Diakonikolas et al [2016a], we obtain near-optimal contamination dependence scaling upto a logarithmic factor as roughly ǫ 2 . Importantly, as emphasized in prior work [Diakonikolas et al, 2016a, Lai et al, 2016 and in stark contrast to other tractable robust estimators, the contamination dependence achieved by our algorithm is completely independent of the dimension of the problem.…”

Section: Re-visiting Illustrative Examplesmentioning

confidence: 88%

“…, z m } to denote the pruned sample. We generalize a similar pruning step from prior works [Diakonikolas et al, 2016a, Lai et al, 2016 to deal with the generalized linear model settings. This in turn further ensures that the subsequent use of the ellipsoid algorithm, terminates in polynomial time.…”

Section: Unified Algorithm and Technical Insightsmentioning

confidence: 99%

“…Considering the low-dimensional setting where d ≪ n, recent works [Diakonikolas et al, 2016a, Lai et al, 2016, Charikar et al, 2016 provide some of the first computationally tractable, provably robust estimators with near-optimal contamination dependence in a variety of settings. Concretely, the paper of Lai et al [2016] considers robust mean and covariance estimation for distributions with appropriately controlled moments, while the work of Diakonikolas et al [2016a], focuses on robust mean and covariance estimation for Gaussians and extends these results to various other models including the mixture of Gaussians. After the initial submission of this manuscript we became aware of independent and concurrent work by Li [2017], addressing similar high-dimensional concerns.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Computationally Efficient Robust Estimation of Sparse Functionals

Du,

Balakrishnan,

Singh

2017

Preprint

View full text Add to dashboard Cite

Many conventional statistical procedures are extremely sensitive to seemingly minor deviations from modeling assumptions. This problem is exacerbated in modern high-dimensional settings, where the problem dimension can grow with and possibly exceed the sample size. We consider the problem of robust estimation of sparse functionals, and provide a computationally and statistically efficient algorithm in the high-dimensional setting. Our theory identifies a unified set of deterministic conditions under which our algorithm guarantees accurate recovery. By further establishing that these deterministic conditions hold with high-probability for a wide range of statistical models, our theory applies to many problems of considerable interest including sparse mean and covariance estimation; sparse linear regression; and sparse generalized linear models.

show abstract

Section: Re-visiting Illustrative Examplesmentioning

confidence: 88%

Section: Unified Algorithm and Technical Insightsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Computationally Efficient Robust Estimation of Sparse Functionals

Du,

Balakrishnan,

Singh

2017

Preprint

View full text Add to dashboard Cite

show abstract

“…Although these mean estimates are robust, the estimation error ∼ √ d (d being the dimension) which is prohibitive in large dimension. There is a recent line of work on robust mean estimation that adapts nicely to high dimension [37,38]. In these results, the mean estimation error is either dimension-independent or very weakly dependent on dimension.…”

Section: Stage Ii-cluster the Ermsmentioning

confidence: 99%

Robust Federated Learning in a Heterogeneous Environment

Ghosh,

Hong,

Yin

et al. 2019

Preprint

View full text Add to dashboard Cite

We study a recently proposed large-scale distributed learning paradigm, namely Federated Learning, where the worker machines are end users' own devices. Statistical and computational challenges arise in Federated Learning particularly in the presence of heterogeneous data distribution (i.e., data points on different devices belong to different distributions signifying different clusters) and Byzantine machines (i.e., machines that may behave abnormally, or even exhibit arbitrary and potentially adversarial behavior). To address the aforementioned challenges, first we propose a general statistical model for this problem which takes both the cluster structure of the users and the Byzantine machines into account. Then, leveraging the statistical model, we solve the robust heterogeneous Federated Learning problem optimally; in particular our algorithm matches the lower bound on the estimation error in dimension and the number of data points. Furthermore, as a by-product, we prove statistical guarantees for an outlier-robust clustering algorithm, which can be considered as the Lloyd algorithm with robust estimation. Finally, we show via synthetic as well as real data experiments that the estimation error obtained by our proposed algorithm is significantly better than the non-Byzantine-robust algorithms; in particular, we gain at least by 53% and 33% for synthetic and real data experiments, respectively, in typical settings.

show abstract

Agnostic Estimation of Mean and Covariance

Cited by 2 publications

References 0 publications

Computationally Efficient Robust Estimation of Sparse Functionals

Computationally Efficient Robust Estimation of Sparse Functionals

Robust Federated Learning in a Heterogeneous Environment

Contact Info

Product

Resources

About