Iterative re‐weighted covariates selection for robust feature selection modelling in the presence of outliers (irCovSel)

Abstract

A new method for feature selective modelling in the presence of outliers is presented. The method is a combination of iterative re-weighted partial least squares and the covariates selection approach. The method relies on iterative down-weighting of the outlying samples prior to estimating the squared covariance for covariates selection. In this way, the outlying samples carrying low weights have minimal influence on the squared covariance estimation, while the inliers have the maximum influence on the squared covariance estimation.The method allows selecting robust features, and models based on such features in general perform better in terms of prediction accuracy (lower error) than selecting features using equal sample weights for all samples. The algorithm description and tests of the method in single and multiple response scenarios are presented. Method performance is also demonstrated on a real spectral data set.

An algorithm for robust multiblock partial least squares predictive modelling

“…As can be noted, the key outlying samples (7, 21, 22, 23, 24, 33) were down‐weighted automatically by the iterative weighting approach. Note also that outliers detected were the same outliers as has been detected in earlier studies using the same data 12,18,23 . The capability of the presented algorithm to detect the same outliers as detected in earlier studies confirms the usefulness of the presented algorithm for robust multiblock data modelling.…”

“…Hence, median‐centring was used as it is less affected in the presence of outlying observations. A recently proposed algorithm for robust feature extraction has also used the median‐centring 18 and has already reported its benefit for handling data containing outlying observations.…”

“…Later, the PRM method, estimated sample weights jointly with Y residuals and scores to deal with both the vertical outliers as well as high‐leverage samples. In the present study, we used adjusted residuals as estimated by Equation () and earlier used in Mishra and Liland 18 where are the OLS residuals, and are the least‐squares fit leverage values.…”

“…As the , the method will become highly aggressive and down‐weighting inliers. It is important to tune using validation approaches like earlier studies 17,18 …”

“…The algorithm handles multi‐response scenarios as well. It uses a similar strategy as RoBoost PLS2 17 and irCovSel, 18 where estimating the samples weights in the multi‐response scenario involves first estimating the sample weights for each response individually and later multiplying the weights for each response to have a single weight per sample. Estimating the weights individually for each response was noted as more robust when the responses have different variances 17 …”