Robust Ridge Regression Based on an M‐estimator

Abstract

Consider the linear regression model y = pol + xp + 6 in the usual notation. It is argued that the class of ordinary ridge estimators obtained by shrinking the least squares estimator by the matrix (X'X + kI)-'X'X is sensitive to outliers in the yvariable. To overcome this problem, we propose a new class of ridgetype M-estimators, obtained by shrinking an M-estimator (instead of the least squares estimator) by the same matrix. Since the optimal value of the ridge parameter k is unknown, we suggest a procedure for choosing it a d a p tively. In a reasonably large scale simulation study with a particular M-estimator, we found that if the conditions are such that the Mestimator is more efficient than the least squares estimator then the corresponding ridgetype M-estimator proposed here is better, in terms of a Mean Squared Error criteria, than the ordinary ridge estimator with k chosen suitably. An example illustrates that the estimators proposed here are less sensitive to outliers in the y-variable than ordinary ridge estimators.

Regularization with Maximum Entropy and Quantum Electrodynamics: The Merg(E) Estimators

“…Maronna (2011, pp. 48-49) discusses the good performance of the RR-MM estimator as well as the drawbacks of other competitors, namely, the estimators proposed by Simpson and Montgomery (1996) and Silvapulle (1991).…”

Linear Regression

“…Then one will obtain ridge-shrinkage estimators~k = W (k)~' Sarkar (94)' for example, considers the ridge-shrinkage restricted least squares estimator W(k),6R' while Silvarob pulle [102] investigates ridge-shrinkage robust estimators W(k)f3 . Then one will obtain ridge-shrinkage estimators~k = W (k)~' Sarkar (94)' for example, considers the ridge-shrinkage restricted least squares estimator W(k),6R' while Silvarob pulle [102] investigates ridge-shrinkage robust estimators W(k)f3 .…”

Robust Liu-type estimator for regression based on M-estimator

“…is proposed by Silvapulle (1991) to overcome this problem, where ̂ is the M-estimator (ME). ME is obtained by minimizing the sum of the function of scaled residuals…”