An Variable Selection Method of the Significance Multivariate Correlation Competitive Population Analysis for Near-Infrared Spectroscopy in Chemical Modeling

Abstract

The high dimensionality of spectral datasets makes it difficult to select the optimal subset of variables. This paper presents a new method for variable selection called the significant multivariate competitive population analysis (SMCPA), Which combines ideas of significant multivariate correlation (SMC) and model population analysis, and employs weighted bootstrap sampling (WBS) and exponential decline function (EDF) competition methods. In this study, the values of SMC distributions are used as an index for evaluating the importance of each wavelength. Then, based on the importance level of each wavelength. SMCPA sequentially selects N subsets of spectral wavelengths by N Monte Carlo sampling in an iterative and competitive procedure. In each sampling run, a fixed ratio of samples is used to build a calibrated partial least-squares model, and then SMC is performed to obtain the score and threshold values. Next, based on the significant multivariate correlation scores, the key variables are selected by two steps: the compulsory selection of exponential decline function and the competitive selection of adaptive weighted sampling. Finally, cross-validation(CV) is applied to select the optimal subset with the lowest root mean square error. This method is tested on three NIR spectral datasets and compared against three highperformance variable selection methods. The experimental results show that the proposed algorithm has the highest efficiency and the best selection effect, and can usually locate the optimal combination of key wavelength variables in a dataset. The evaluation result after PLS modeling is also the best. INDEX TERMS Spectrochemical analysis, variable selection, the significant multivariate correlation, weighted bootstrap sampling, model population analysis, monte Carlo sampling, analytical techniques, partial least squares method.

A Bootstrapping Soft Shrinkage Approach and Interval Random Variables Selection Hybrid Model for Variable Selection in Near-Infrared Spectroscopy

“…The value of RMSEC for BOSS-IRVS with six added variables has low RMSEC and high RMSEP compared with the BOSS-IRVS with three added variables. The variables around 1104-1400nm can be selected by all methods which indicate the importance of this region which corresponds to the first overtone of the O-H stretch bond vibration [7]. The VCPA-IRIV select other variables in intervals around 1800 and between 2200 and 2400.…”

“…Second, fisher optimal subspace shrinkage (FOSS) [6] that splits variables into some intervals by the information from regression coefficients PLS model, then the weighted block bootstrap sampling (WBBS) is used to select intervals, and the mean of the absolute values of regression coefficients of the corresponding interval determines the weights of sub-intervals. Third, significant multivariate competitive population analysis (SMCPA) that combines the ideas of substantial multivariate correlation (SMC) and MPA, and employs WBS is an improved version of bootstrap sampling with different weights on sampling objects and exponential decline function (EDF) competition method used to force the elimination of uninformative or redundancy variables [7]. For corn and wheat protein datasets, both methods select informative intervals including the BOSS.…”

A hybrid variable selection method combining Fisher's linear discriminant combined population analysis and an improved binary cuckoo search algorithm

“…In recent years, model population analysis (MPA) has become very popular in the study of variable selection methods. 13,14 Various variable selection approaches based on the MPA algorithm have been developed, including bootstrapping soft shrinkage (BOSS), 15 stabilized bootstrapping soft shrinkage approach (SBOSS), 16 BOSS and interval random variable selection(BOSS-IRVS), 17 variable iterative space shrinkage approach (VISSA), 18 interval combination optimization (ICO), 19 significant multivariate competitive population analysis (SMCPA), 20 etc. These methods construct multiple models by using different subsets of variables and select the best combination of variables by comparing the performance of the models.…”