Cellwise robust M regression

Filzmoser, Peter; Höppner, Sebastiaan; Ortner, Irene; Serneels, Sven; Verdonck, Tim

doi:10.1016/j.csda.2020.106944

Cited by 23 publications

(11 citation statements)

References 22 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is already a generalization in the literature from SVD13 to MI (Arciniegas‐Alarcón et al., 2014b), so this generalization could be studied with the robust extensions recommended here using other parameters of interest in multi‐environment experiments (Yan, 2014). Recent developments in statistics, such as the PCA, allowing for missingness and cellwise and row‐wise outliers, macroPCA (Hubert et al., 2019), and the cellwise robust M regression (Filzmoser et al., 2020) can be compared with the results obtained here, but in matrices and multivariate situations.…”

Section: Discussionmentioning

confidence: 80%

Missing‐value imputation using the robust singular‐value decomposition: Proposals and numerical evaluation

et al. 2021

View full text Add to dashboard Cite

A common problem in the analysis of data from multi‐environment trials is imbalance caused by missing observations. To get around this problem, Yan proposed a method for imputing the missing values based on the singular‐value decomposition (SVD) of a matrix. However, this SVD can be affected by outliers and produce low quality imputations. In this article, we propose four extensions of the Yan method that are resistant to outliers, replacing the standard SVD method with four robust SVD extensions. We evaluate these methods, using exclusively numerical criteria in a simulation study and in a cross‐validation study based on real data. We conclude that in the presence of outliers, the standard SVD method should not be used; instead, the best alternatives are the robust SVD methods based on sub‐sampling when the percentage of contamination is less than 2% following a completely random missing data mechanism. In any other case, methods that either minimize the L2 norm or that involve L1 regressions are preferable.

show abstract

Section: Discussionmentioning

confidence: 80%

Missing‐value imputation using the robust singular‐value decomposition: Proposals and numerical evaluation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Our statement is however correct and in line with the pessimistic BDP perspective which only considers worst cases, i.e., harmful outliers that, depending on the concrete application and BDP notion, may not even be likely in practice. When having simulated data and a random cell-wise outlier scheme, it is extremely unlikely that such columnwise outliers that make the true model irretrievable would appear, so this situation should at most very rarely occur in a random simulation as for example done in Filzmoser et al [2020a] where first a set of instances is selected and for each of these instances, a random fraction of cells are contaminated. We do not dare to make a definite statement whether column-wise outliers can occur in practice, but, for example, Filzmoser et al [2020a] identify the column values with measurements made by a specific sensor, so if this sensor is corrupted across a whole study, one indeed would have column-wise outliers.…”

Section: Evidently It Turns Out That the Fraction Of Cell-wise Outlie...mentioning

confidence: 99%

“…As already pointed out, even in the presence of a clean regressor matrix, no regression (or other supervised machine learning method) procedure has a chance to infer the correct model if too many responses are contaminated. Even worse, if the model cannot be inferred, the contaminated responses cannot be suitably imputed in contrast to outlying cells as for example in Filzmoser et al [2020a].…”

Section: Proof I)+ii) Obvious and Already Partially Discussed Earliermentioning

confidence: 99%

“…This causes even the sophisticated robust methods that are tailored for row-wise outliers to break down since they can only handle at most a fraction of contaminated instances of one half. Nevertheless, there are already some algorithms that can cope with cell-wise outliers like the robust clustering algorithm that trims outlying cells from García-Escudero et al [2021], cell-wise robust location and scatter matrix estimation algorithms (Agostinelli et al [2015], Leung et al [2017]) and several regression approaches (Bottmer et al [2021], Leung et al [2016], Filzmoser et al [2020a]). There are also notable algorithms for detecting and imputing cell-wise outliers like the DDC from Rousseeuw and Van Den Bossche [2018] or the cellHandler from Raymaekers and Rousseeuw [2019].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Trimming Stability Selection increases variable selection robustness

Werner¹

2021

Preprint

View full text Add to dashboard Cite

Contamination can severely distort an estimator unless the estimation procedure is suitably robust. This is a well-known issue and has been addressed in Robust Statistics, however, the relation of contamination and distorted variable selection has been rarely considered in literature. As for variable selection, many methods for sparse model selection have been proposed, including Stability Selection which is a meta-algorithm based on some variable selection algorithm in order to immunize against particular data configurations. We introduce the variable selection breakdown point that quantifies the number of cases resp. cells that have to be contaminated in order to let no relevant variable be detected. We show that particular outlier configurations can completely mislead model selection and argue why even cell-wise robust methods cannot fix this problem. We combine the variable selection breakdown point with resampling, resulting in the Stability Selection breakdown point that quantifies the robustness of Stability Selection. We propose a trimmed Stability Selection which only aggregates the models with the lowest in-sample losses so that, heuristically, models computed on heavily contaminated resamples should be trimmed away. We provide a short simulation study that reveals both the potential of our approach as well as the fragility of variable selection, even for an extremely small cell-wise contamination rate.

show abstract

“…Methods assuming a cellwise contamination are, however, still not that well investigated. For example, in a regression setting assuming n > p, MM regression was considered in Filzmoser, Höppner, Ortner, Serneels, and Verdonck (2020), a three step procedure based on S-estimators in Leung, Zhang, and Zamar (2016) shooting S-estimator in Öllerer, Alfons, and Croux (2016), combining ideas from simple S-regression with the 'shooting algorithm', which is a coordinate descent algorithm. The shooting S-estimator was recently extended to the high-dimensional setting in Bottmer, Croux, and Wilms (2020).…”

Section: Cellwise Contaminationmentioning

confidence: 99%

Robust linear regression for high‐dimensional data: An overview

Filzmoser

Nordhausen²

2020

WIREs Computational Stats

Self Cite

View full text Add to dashboard Cite

Digitization as the process of converting information into numbers leads to bigger and more complex data sets, bigger also with respect to the number of measured variables. This makes it harder or impossible for the practitioner to identify outliers or observations that are inconsistent with an underlying model. Classical least-squares based procedures can be affected by those outliers. In the regression context, this means that the parameter estimates are biased, with consequences on the validity of the statistical inference, on regression diagnostics, and on the prediction accuracy. Robust regression methods aim at assigning appropriate weights to observations that deviate from the model. While robust regression techniques are widely known in the lowdimensional case, researchers and practitioners might still not be very familiar with developments in this direction for high-dimensional data. Recently, different strategies have been proposed for robust regression in the high-dimensional case, typically based on dimension reduction, on shrinkage, including sparsity, and on combinations of such techniques. A very recent concept is downweighting single cells of the data matrix rather than complete observations, with the goal to make better use of the model-consistent information, and thus to achieve higher efficiency of the parameter estimates. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Robust Methods Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data Statistical and Graphical Methods of Data Analysis > Dimension Reduction

show abstract

Cellwise robust M regression

Cited by 23 publications

References 22 publications

Missing‐value imputation using the robust singular‐value decomposition: Proposals and numerical evaluation

Missing‐value imputation using the robust singular‐value decomposition: Proposals and numerical evaluation

Trimming Stability Selection increases variable selection robustness

Robust linear regression for high‐dimensional data: An overview

Contact Info

Product

Resources

About