Ordinary Least Squares with Laboratory Calibrations: A Practical Way to Show Students that This Fitting Model may Easily Yield Biased Results When Used Indiscriminately

Abstract

Analytical calibration using ordinary least squares (OLS) is the most widely applied response function for calibration in all type of laboratories. However, this calibration function is not always the most adequate and its indiscriminant use can lead to obtain biased estimates of unknowns. Students need to be taught about the practical requirements needed to obtain good results with OLS and when this fitting method is not accurate. Different experimental calibration curves were obtained in laboratory sessions using two common instrumental techniques: chromatography and atomic absorption spectrometry. After discussion seminars evaluating the data obtained by students, they were able to understand that linear fitting was not the most accurate model using atomic absorption spectrometry and a quadratic fitting provided most accurate estimates. Linearity was confirmed in chromatographic calibrations, but data presented heteroscedasticity, which is very common in calibrations done in chemical and biological analyses. A simple experiment was applied to show students how the use of the regression coefficients obtained by OLS with heteroscedastic data lead to highly biased estimates near the quantification limits of the calibration curve. The results obtained allowed to show students that, despite being widely used, OLS is not the most adequate fitting model to obtain accurate and precise results with many calibration methods routinely used in chemical and biological laboratories.

Estimating Detection Limits in Chromatography from Calibration Data: Ordinary Least Squares Regression vs. Weighted Least Squares

“…However, despite the fact that many researchers often do not take it into account, heteroscedasticity is more frequent than might be expected in experimental sciences. Many analytical methods yield non-constant variances over the calibration range [8,26,27,[30][31][32][33][34], as was the case with the calibrations evaluated in the present study ( Figure 1). In these conditions, the absolute errors of the instrument tend to be proportional to the concentrations, and the relative standard deviation is the constant parameter across the curve instead of the standard deviation [33,[35][36][37][38].…”

“…Previous studies have found that the variances obtained at low concentrations with WLS are significantly reduced when compared to OLS, and that precision loss with OLS calculations can be as high as one order of magnitude in the lower range of the calibration curves [32,[35][36][37]. The results obtained in the present study show that in 17 calibrations s b0 values determined by WLS were significantly smaller (2-23 times, p < 0.05, Fisher F-test) than by OLS (Tables 1-4), which agrees with the results obtained by other studies comparing OLS and WLS with experimental calibrations involving heteroscedastic data [8,26,27,47].…”

The inadequate use of the determination coefficient in analytical calibrations: How other parameters can assess the goodness‐of‐fit more adequately

“…The most used linear regression function, OLS, is based on the minimization of the SSE term (also known as residuals) because this model was developed with the assumption that absolute errors of the dependent variable (measured as SD or variance, SD 2 ) are constant all along the range studied (homoscedasticity). However, in analytical and bioanalytical calibrations the most common situation is that absolute errors are not constant (heteroscedasticity) and the parameter that remains approximately constant is the relative error (RE; RSD) [6,[8][9][10][11][12][13]. In this situation, OLS regression overestimates the effect of calibrators at high concentration ranges, and the higher variations at this level have a much greater influence on R 2 than small deviations present at low ranges [14].…”

“…It has been reported that when the data have a proportional error, neglect of weighting can increase the uncertainty by a factor ≥10 at the lower concentration level [15,16]. For these reasons, it has been reported that OLS should not be used when samples are expected to be determined close to the LOQs [1,[6][7][8][9]12,13,15,[17][18][19], as is very common in trace analysis.…”

“…In this situation, the lower concentration standards become more important. In all the studies that have compared OLS with WLS regression with heteroscedastic calibrations, it has been found that significantly biased results with OLS regression were obtained at low calibration levels [1,12,13,17,20], which are solved applying WLS. However, at a certain level above the LOQ of the method, both linear regression models tend to yield equivalent results.…”