Variable selection in support vector regression using angular search algorithm and variance inflation factor

Abstract

Here, we combine angular search algorithm and variance inflation factor (ASA-VIF) with support vector regression (SVR) (ASA-VIF-SVR) to estimate total acid number (TAN), basic nitrogen content (BNC), and sulfur content (SC) in Brazilian crude oils. To prevent the interference of outliers, we further developed a strategy for outlier identification and applied it to nonlinear models based on RMSE (root mean square error). ASA-VIF-SVR was applied to near-and mid-infrared spectroscopy (NIR and MIR) and hydrogen nuclear magnetic resonance (1 H NMR) spectroscopy data available in a range of 93-194 samples. The models were evaluated for accuracy (root mean square error of calibration [RMSEC] and root mean square error of prediction [RMSEP]) and linearity (coefficient of determination, R 2). The removal of outliers increased accuracy and linearity of our models. The ASA-VIF model for TAN, BNC, and SC selected 0.37%, 0.93%, and 0.30% of variables from full NIR spectra; 0.21%, 0.27%, and 0.21% from full MIR; and 0.20%, 0.42%, and 0.15% from full 1 H NMR. In most cases, the best results were obtained with variable selection compared with the full dataset. Also, 1 H NMR generated more accurate and linear models with RMSEP and R 2 p of 0.0071 wt% and 0.86 for BNC and 0.0623 wt% and 0.79 for SC. TAN showed a better MIR result with RMSEP of 0.1426 mg KOH g-1 and R 2 p of 0.47. The most important region for 1 H NMR and MIR was the one with the largest quantity of unpaired electrons (aromatic region).

Variable selection by permutation applied in support vector regression models

“…Another variable selection is the angular search algorithm and variance inflation factor (ASA‐VIF). This variable selection method has as main objective to eliminate correlated variables, remaining only those that do not have shared information 18 . The chemical spectra have a vast number of variables that have correlated information 19 .…”

“…Thus, the number of variables can decrease drastically. The SVR has a rank limitation due to the number of samples, generally much lower than the number of variables 18 . Thus, the drastic reduction of variables provides an optimization of the SVR modeling, associated with ASA.…”

“…Afterwards, VIF is applied to eliminate variables that still have high multicollinearity. Then, random sample subsets are created to calculate the smallest root mean square error of cross validation (RMSECV) 18 . However, ASA‐VIF is time‐consuming and hard processing, mainly for large matrix, due to the correlation and multicollinearity calculation performed.…”

“…There are several variables selection methods applied in SVR, such as CARS‐SVR and UVE‐SVR in NIR spectroscopy data, 15 Firefly algorithm and GA‐SVR in ultraviolet (UV) spectral data, 16 ASA‐VIF‐SVR in mid infrared (MIR), near infrared (NIR), and 1 H NMR spectroscopy, 18 and GA‐SVR in NMR spectroscopy 17 . However, SPA and NISPA in SVR are still poorly applied.…”

“…This variable selection method has as main objective to eliminate correlated variables, remaining only those that do not have shared information. 18 The chemical spectra have a vast number of variables that have correlated information. 19 Thus, the number of variables can decrease drastically.…”