Covariance Estimation: Optimal Dimension-free Guarantees for Adversarial Corruption and Heavy Tails

Abstract

We provide an estimator of the covariance matrix that achieves the optimal rate of convergence (up to constant factors) in the operator norm under two standard notions of data contamination: We allow the adversary to corrupt an η-fraction of the sample arbitrarily, while the distribution of the remaining data points only satisfies that the L pmarginal moment with some p 4 is equivalent to the corresponding L 2 -marginal moment. Despite requiring the existence of only a few moments, our estimator achieves the same tail estimates as if the underlying distribution were Gaussian. As a part of our analysis, we prove a dimension-free Bai-Yin type theorem in the regime p > 4.

Statistically Optimal Robust Mean and Covariance Estimation for Anisotropic Gaussians

“…provided that N c 2 (r(Σ)+log(1/δ)). When adversarial contamination is allowed, the sharpest known dimension-free result is implied by the bound of Abdalla and the second author of this paper [AZ22]. Their work suggests a trimmed-mean-based estimator that achieves the rate…”

“…This step follows from existing results. In particular, in the Gaussian case Proposition 6 in [AZ22] provides an estimator ω satisfying Σ /4 ω 4 Σ whenever N c(r(Σ) + log(1/δ)), where c > 0 is an absolute constant. We also need to estimate Tr(Σ).…”

Robust Estimation of Covariance Matrices: Adversarial Contamination and Beyond

“…INTRODUCTION. (2016), Giulini (2015), Fan et al (2016), Abdalla and Zhivotovskiy (2022), Oliveira and Rico (2022) as well as Minsker (2018); Minsker and Wei (2020).…”

“…The survey by Ke et al (2019) contains a more detailed overview of the recent progress. Contributions by theoretical computer scientists have introduced a range of new ideas leading to theoretically optimal estimators in the adversarial contamination framework; a small subsample of works includes Lai et al (2016); Diakonikolas et al (2021Diakonikolas et al ( , 2017; Cheng et al (2019) as well as the excellent survey ; the very recent works by Abdalla and Zhivotovskiy (2022), Oliveira and Rico (2022) describe estimators that achieve sharpest possible bounds. While several proposed approaches, including the very recent works by Abdalla and Zhivotovskiy (2022), Oliveira and Rico (2022), result in optimal with respect to the contamination proportion and dependence on the dimension factors estimators, the corresponding algorithms are either not computationally feasible or not user-friendly, as they are often sensitive to the choice of "absolute constants" appearing in the tuning parameters, require preliminary robust mean estimation, or assume that (typically unknown) parameters such as the contamination proportion ε are given as an input; other works focus only on the bounds with respect to the Frobenius norm, while we are interested in the error measured in the operator norm as well.…”