The prediction error in CLS and PLS: the importance of feature selection prior to multivariate calibration

Abstract

Classical least squares (CLS) and partial least squares (PLS) are two common multivariate regression algorithms in chemometrics. This paper presents an asymptotically exact mathematical analysis of the mean squared error of prediction of CLS and PLS under the linear mixture model commonly assumed in spectroscopy. For CLS regression with a very large calibration set the root mean squared error is approximately equal to the noise per wavelength divided by the length of the net analyte signal vector. It is shown, however, that for a finite training set with n samples in p dimensions there are additional error terms that depend on r 2 p 2 =n 2 , where r is the noise level per co-ordinate.Therefore in the 'large p-small n' regime, common in spectroscopy, these terms can be quite large and even dominate the overall prediction error. It is demonstrated both theoretically and by simulations that dimensional reduction of the input data via their compact representation with a few features, selected for example by adaptive wavelet compression, can substantially decrease these effects and recover the asymptotic error. This analysis provides a theoretical justification for the need to perform feature selection (dimensional reduction) of the input data prior to application of multivariate regression algorithms.

Partial least squares, Beer's law and the net analyte signal: statistical modeling and analysis

“…Our analysis is limited to the cases of either a finite error-free setting or a noisy but infinite population setting. While many simulations have studied the effects of various parameters on PLS and other competing algorithms in the presence of a finite and noisy training set, a theoretical statistical analysis in this case is still an open research problem [14].…”

“…1. The analysis of PLS with a finite and noisy calibration set is considered in Reference [14]. We first consider a system with a single component, for which we assume input data of the form…”

“…The regression vector computed by PLS is simply the result of an ordinary least squares regression on these projections. This is equivalent to minimizing (14) under the restriction that r PLS 2 Spanfw w j g k j¼0 .…”

“…1. The more complicated (and more interesting) theoretical analysis of PLS predictions based on a finite and noisy training set will be published separately [14].…”

Wavelength selection using the measure of topological relevance on the self‐organizing map

“…One standard solution is to rely on full-spectrum methods for linear dimension reduction coupled with linear regression: the basic formulations of principal components regression (PCR) and partial least-squares regression (PLSR) are reference models. The natural refinement of such an approach benefits from a preliminary selection of relevant wavelength ranges [2] as performed by one of the many available techniques (e.g. see References [3][4][5][6][7][8][9][10][11]).…”