Partial least squares, Beer's law and the net analyte signal: statistical modeling and analysis

Abstract

Partial least squares (PLS) is one of the most common regression algorithms in chemistry, relating input-output samples (x i , y i ) by a linear multivariate model. In this paper we analyze the PLS algorithm under a specific probabilistic model for the relation between x and y. Following Beer's law, we assume a linear mixture model in which each data sample (x, y) is a random realization from a joint probability distribution where x is the sum of k components multiplied by their respective characteristic responses, and each of these components is a random variable. We analyze PLS on this model under two idealized settings: one is the ideal case of noise-free samples and the other is the case of an infinite number of noisy training samples. In the noise-free case we prove that, as expected, the regression vector computed by PLS is, up to normalization, the net analyte signal. We prove that PLS computes this vector after at most k iterations, where k is the total number of components. In the case of an infinite training set corrupted by unstructured noise, we show that PLS computes a final regression vector which is not in general purely proportional to the net analyte signal vector, but has the important property of being optimal under a mean squared error of prediction criterion. This result can be viewed as an asymptotic optimality of PLS in the limit of a very large but finite training set.

An Accurate Quantitative Analysis of Polymorphic Content by Chemometric X-ray Powder Diffraction

“…In addition, PLS regression combines the basic functions of a regressing model, PCA, and a canonical correlation analysis. 7,[26][27][28][29] PLS also has the advantage that the precision of the model parameters is improved by increasing the number of relevant variables and observations. For our purposes, PLS regression is a useful algorithm because the scores of the spectral space (Xvariable) are always correlated to those of the concentration space (Y-variable).…”

The prediction error in CLS and PLS: the importance of feature selection prior to multivariate calibration

“…1. The following theorem, proven in Reference [24], characterizes the limiting behavior of PLS as n ! 1 on inputs of the form (1) and (2).…”

“…Much effort was put forth to elucidate the PLS algorithm from a statistical point of view [21][22][23][24], although a theory for the performance of PLS under the linear mixture model with a finite and noisy training set was not considered. In terms of theoretical formulae for the expected mean squared error of prediction, most attention has been devoted to the study of other multivariate regression algorithms such as the generalized least squares and best linear predictor algorithms and not of the more common CLS and PLS algorithms.…”

The chemometrics revolution re‐examined

“…Gradually, by circa 2005, a few other statisticians began to study PLS and to publish their findings in chemometrics journals. 52,53 However, even in 2009, many statisticians were not willing to embrace algorithmic approaches to analysis 54 ; the drivers for the separation still show up in the statistics literature in comments on the perspective of Breiman discussed earlier. Friedman, in writing his own retrospective, regards the dissociation with chemometrics in the 1990s as a missed opportunity for statistics.…”