Partial robust M-regression

Abstract

Partial Least Squares (PLS) is a standard statistical method in chemometrics. It can be considered as an incomplete, or "partial", version of the Least Squares estimator of regression, applicable when high or perfect multicollinearity is present in the predictor variables. The Least Squares estimator is well-known to be an optimal estimator for regression, but only when the error terms are normally distributed. In absence of normality, and in particular when outliers are in the data set, other more robust regression estimators have better properties. In this paper a "partial" version of M-regression estimators will be defined. If an appropriate weighting scheme is chosen, partial M-estimators become entirely robust to any type of outlying points. It is shown that robust M-regression outperforms existing methods for robust PLS regression in terms of statistical precision and computational speed, while keeping the robustness properties. The method is applied to a data set consisting of EPXMA spectra of archffiological glass vessels. This data set contains several outliers, and the advantages of Partial Robust M-regression are illustrated. Applying Partial Robust M-regression yields much smaller prediction errors for noisy calibration samples than PLS. On the other hand, if the data follow perfectly well a normal model, the loss in efficiency to be paid for is very small.

Robust methods for multivariate data analysis

“…In situations with 10% bad leverage points PRM and RSIMPLS performed almost similar and clearly outperformed PLS. When comparing the computation time for PRM and SIMPLS it turned out that the computation time for RSIMPLS was consistently substantially higher than for PRM both for increasing number of observations and increasing number of predictor variables [90].…”

“…PLS1) and are not resistant to high [90] leverage points since the weights only depends on the residuals after each step [89]. Different weight functions as well as different tuning constant for the same weight function can give different results, which may make the methods less attractive [79,87].…”

“…Both RSIMPLS and RSIMCD are equivariant for translation and orthogonal transformations in x and y [89]. Recently, Serneels et al [90] proposed a method, partial robust M-regression (PRM), for robust regression based on GM-estimators. PRM uses continuous weights, resulting in a gradual down-weighting of outliers according to their degree of outlyingness.…”

A robust moving average iterative weighting method to analyze the effect of outliers on the response surface design

“…Nguyena (2010) studied outlier detection and proposed new least trimmed squares approximate. Recently a "partial" version of the M-estimator based on the "fair" ψ function and an appropriate weighting scheme was proposed by Serneels et al (2005). The authors claim that the partial robust M-regression outperforms existing methods for robust partial least square regression.…”

Sparse partial robust M regression

“…Manifold proposals to robustify PLS have been discussed of which a good overview is given in Filzmoser et al (2009). One of the most widely applied robust alternatives to PLS is partial robust M regression (Serneels et al, 2005). Likely its popularity is due to the fact that it provides a fair tradeoff between statistical robustness with respect to both vertical outliers and leverage points on the one hand and statistical and computational efficiency on the other hand.…”