Robust estimation of multivariate location and scatter in the presence of cellwise and casewise contamination

Agostinelli, Claudio; Leung, Andy; Yohai, Vı́ctor J.; Zamar, Ruben H.

doi:10.1007/s11749-015-0450-6

Cited by 87 publications

(116 citation statements)

References 17 publications

Supporting

Mentioning

112

Contrasting

Order By: Relevance

“…We have shown that exact affine equivariance must be lost, but it is a reasonable price to be paid in order to achieve an arbitrarily high breakdown for the resulting trimmed estimators. This conclusion parallels similar findings in other situations where contamination produces only a minority of “good” observations, as in the case of cellwise contamination (see, e.g., Farcomeni, , ; Agostinelli, Leung, Yohai, & Zamar, ; Rousseeuw & Van den Bossche, ). We also support the use of adaptive trimming schemes, in order to explore the effect of different levels of trimming and to find a sensible trade‐off between robustness and efficiency.…”

Section: Discussionsupporting

confidence: 88%

Wild adaptive trimming for robust estimation and cluster analysis

Cerioli

Farcomeni

Riani

2018

Scandinavian J Statistics

View full text Add to dashboard Cite

Trimming principles play an important role in robust statistics. However, their use for clustering typically requires some preliminary information about the contamination rate and the number of groups. We suggest a fresh approach to trimming that does not rely on this knowledge and that proves to be particularly suited for solving problems in robust cluster analysis. Our approach replaces the original K‐population (robust) estimation problem with K distinct one‐population steps, which take advantage of the good breakdown properties of trimmed estimators when the trimming level exceeds the usual bound of 0.5. In this setting, we prove that exact affine equivariance is lost on one hand but, on the other hand, an arbitrarily high breakdown point can be achieved by “anchoring” the robust estimator. We also support the use of adaptive trimming schemes, in order to infer the contamination rate from the data. A further bonus of our methodology is its ability to provide a reliable choice of the usually unknown number of groups.

show abstract

Section: Discussionsupporting

confidence: 88%

Wild adaptive trimming for robust estimation and cluster analysis

Cerioli

Farcomeni

Riani

2018

Scandinavian J Statistics

View full text Add to dashboard Cite

show abstract

“…The devastating effect of combined cellwise and casewise outliers on standard robust estimates has recently been discussed in Agostinelli et al (2015) who introduced a second generation of robust estimates that can deal simultaneously with these two DQ issues. These procedures can also be used to summarize and analyse AD.…”

Section: Daniel L Oberski (Utrecht University)mentioning

confidence: 99%

Statistical Challenges of Administrative and Transaction Data

Hand

2018

Journal of the Royal Statistical Society Series A: Statistics in Society

100

View full text Add to dashboard Cite

Summary Administrative data are becoming increasingly important. They are typically the side effect of some operational exercise and are often seen as having significant advantages over alternative sources of data. Although it is true that such data have merits, statisticians should approach the analysis of such data with the same cautious and critical eye as they approach the analysis of data from any other source. The paper identifies some statistical challenges, with the aim of stimulating debate about and improving the analysis of administrative data, and encouraging methodology researchers to explore some of the important statistical problems which arise with such data.

show abstract

“…However, as for the original estimate, computation times are not feasible for larger numbers of variables. A very recent approach by Agostinelli et al [2015] flags cellwise outliers as missing values and applies afterwards a rowwise robust method that can deal with missing values. By this, it can deal with cellwise and rowwise outliers at the same time, but again, computation for high-dimensions is not achievable.…”

Section: Introductionmentioning

confidence: 99%

Robust High-Dimensional Precision Matrix Estimation

Öllerer

Croux

2015

Modern Nonparametric, Robust and Multivariate Methods

View full text Add to dashboard Cite

The dependency structure of multivariate data can be analyzed using the covariance matrix Σ. In many fields the precision matrix Σ −1 is even more informative. As the sample covariance estimator is singular in high-dimensions, it cannot be used to obtain a precision matrix estimator. A popular highdimensional estimator is the graphical lasso, but it lacks robustness. We consider the high-dimensional independent contamination model. Here, even a small percentage of contaminated cells in the data matrix may lead to a high percentage of contaminated rows. Downweighting entire observations, which is done by traditional robust procedures, would then results in a loss of information. In this paper, we formally prove that replacing the sample covariance matrix in the graphical lasso with an elementwise robust covariance matrix leads to an elementwise robust, sparse precision matrix estimator computable in high-dimensions. Examples of such elementwise robust covariance estimators are given. The final precision matrix estimator is positive definite, has a high breakdown point under elementwise contamination and can be computed fast.

show abstract

Robust estimation of multivariate location and scatter in the presence of cellwise and casewise contamination

Cited by 87 publications

References 17 publications

Wild adaptive trimming for robust estimation and cluster analysis

Wild adaptive trimming for robust estimation and cluster analysis

Statistical Challenges of Administrative and Transaction Data

Robust High-Dimensional Precision Matrix Estimation

Contact Info

Product

Resources

About