A hybrid optimization method for sample partitioning in near-infrared analysis

Chen, Wei-Hao; Chen, Huazhou; Feng, Qinghua; Mo, Lina; Hong, Shaoyong

doi:10.1016/j.saa.2020.119182

Cited by 23 publications

(10 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The egg samples were divided into a calibration set (2/3) and a prediction set (1/3) by using sample set partitioning based on the joint X-Y distances (SPXY) algorithm [ 26 ]. The spectral data contain a large amount of useful information related to S-ovalbumin, as well as some redundant and interfering information.…”

Section: Methodsmentioning

confidence: 99%

Development of Simplified Models for Non-Destructive Hyperspectral Imaging Monitoring of S-ovalbumin Content in Eggs during Storage

Yao

Sun

Cheng

et al. 2022

Foods

View full text Add to dashboard Cite

S-ovalbumin content is an indicator of egg freshness and has an important impact on the quality of processed foods. The objective of this study is to develop simplified models for monitoring the S-ovalbumin content of eggs during storage using hyperspectral imaging (HSI) and multivariate analysis. The hyperspectral images of egg samples at different storage periods were collected in the wavelength range of 401–1002 nm, and the reference S-ovalbumin content was determined by spectrophotometry. The standard normal variate (SNV) was employed to preprocess the raw spectral data. To simplify the calibration models, competitive adaptive reweighted sampling (CARS) was applied to select feature wavelengths from the whole spectral range. Based on the full and feature wavelengths, partial least squares regression (PLSR) and least squares support vector machine (LSSVM) models were developed, in which the simplified LSSVM model yielded the best performance with a coefficient of determination for prediction (R2P) of 0.918 and a root mean square error for prediction (RMSEP) of 7.215%. By transferring the quantitative model to the pixels of hyperspectral images, the visualizing distribution maps were generated, providing an intuitive and comprehensive evaluation for the S-ovalbumin content of eggs, which helps to understand the conversion of ovalbumin into S-ovalbumin during storage. The results provided the possibility of implementing a multispectral imaging technique for online monitoring the S-ovalbumin content of eggs.

show abstract

Section: Methodsmentioning

confidence: 99%

Development of Simplified Models for Non-Destructive Hyperspectral Imaging Monitoring of S-ovalbumin Content in Eggs during Storage

Yao

Sun

Cheng

et al. 2022

Foods

View full text Add to dashboard Cite

show abstract

“…The spectral data x and HA concentration y affect the modeling results; therefore, when considering the distance between samples, the same importance is given to the distance between the spectral data x and HA concentration y in the space to ensure that the sample distribution is maximally represented and the multidimensional vector space is effectively covered. 22,23 The specific division formulas are as follows:…”

Section: Sample Set Divisionmentioning

confidence: 99%

Quantitative analysis of the hexamethylenetetramine concentration in a hexamethylenetetramine–acetic acid solution using near infrared spectroscopy: A comprehensive study on preprocessing methods and variable selection techniques

Chao,

Qian,

Wang

et al. 2024

Journal of Near Infrared Spectroscopy

View full text Add to dashboard Cite

Hexamethylenetetramine (HA) is widely used as a raw material in the medical, chemical, industrial, and military industries, and the fast and quantitative analysis of HA is important for manufacturing processes in these fields. Owing to its efficiency, low cost, nondestructive testing, and convenience, near infrared (NIR) spectroscopy is a powerful technique for quantitatively analyzing the HA concentration in HA–acetic acid (HAc) solutions, demonstrating application potential in the production of hexogen and octogen. A series of preprocessing algorithms and variable selection methods were studied to improve the detection accuracy of the NIR spectroscopic calibration. Forty-six different combinations of standard normal variation (SNV), multiplicative signal correction (MSC), first derivative (1stDer), second derivative (2ndDer), and discrete wavelet transform (DWT) were screened. The effects of four variable selection methods (successive projection algorithm (SPA), uninformed variable elimination (UVE), competitive adaptive reweighted sampling (CARS), and multiverse optimization (MVO)) were compared. Finally, a model (SPXY-SNV-1stDer-DWT-MVO-RF) was developed by combining sample set portioning based on the joint x–y distance (SPXY) algorithm with the random forest (RF) calibration model, and MVO was combined with the NIR technique for the first time. The model achieved a coefficient of determination for the calibration set (R2), root mean square error of the calibration set (RMSEC), coefficient of determination for the prediction set (r2), and root mean square error of the prediction set (RMSEP) of 1.000, 0.04%, 0.999, and 0.05%, respectively. This study demonstrated the novelty and feasibility of HA quantitative detection by NIR spectroscopy and provided valuable insights for optimizing quantitative analysis models by optimizing algorithms, indicating the great application potential of NIR spectroscopy in related fields.

show abstract

“…Then, it produces new samples by random insertion on the connecting lines. The connection and insertion operation can reduce the imbalance of sample space and simultaneously prevent the over-fitting phenomenon by suppressing too large repetition of the original minority samples (Fernández et al, 2018;Chen et al, 2021). The schematic diagram for generating new samples by the SMOTE algorithm is shown in Figure 1.…”

Section: The Principle Of Smotementioning

confidence: 99%

Electricity Theft Detection in Power Consumption Data Based on Adaptive Tuning Recurrent Neural Network

et al. 2021

Self Cite

View full text Add to dashboard Cite

Electricity theft behavior has serious influence on the normal operation of power grid and the economic benefits of power enterprises. Intelligent anti-power-theft algorithm is required for monitoring the power consumption data to recognize electricity power theft. In this paper, an adaptive time-series recurrent neural network (TSRNN) architecture was built up to detect the abnormal users (i.e., the electricity theft users) in time-series data of the power consumption. In fusion with the synthetic minority oversampling technique (SMOTE) algorithm, a batch of virtual abnormal observations were generated as the implementation for training the TSRNN model. The power consumption record was characterized with the sharp data (ARP), the peak data (PEA), and the shoulder data (SHO). In the TSRNN architectural framework, a basic network unit was formed with three input nodes linked to one hidden neuron for extracting data features from the three characteristic variables. For time-series analysis, the TSRNN structure was re-formed by circulating the basic unit. Each hidden node was designed receiving data from both the current input neurons and the time-former neuron, thus to form a combination of network linking weights for adaptive tuning. The optimization of the TSRNN model is to automatically search for the most suitable values of these linking weights driven by the collected and simulated data. The TSRNN model was trained and optimized with a high discriminant accuracy of 95.1%, and evaluated to have 89.3% accuracy. Finally, the optimized TSRNN model was used to predict the 47 real abnormal samples, resulting in having only three samples false predicted. These experimental results indicated that the proposed adaptive TSRNN architecture combined with SMOTE is feasible to identify the abnormal electricity theft behavior. It is prospective to be applied to online monitoring of distributed analysis of large-scale electricity power consumption data.

show abstract

A hybrid optimization method for sample partitioning in near-infrared analysis

Cited by 23 publications

References 33 publications

Development of Simplified Models for Non-Destructive Hyperspectral Imaging Monitoring of S-ovalbumin Content in Eggs during Storage

Development of Simplified Models for Non-Destructive Hyperspectral Imaging Monitoring of S-ovalbumin Content in Eggs during Storage

Quantitative analysis of the hexamethylenetetramine concentration in a hexamethylenetetramine–acetic acid solution using near infrared spectroscopy: A comprehensive study on preprocessing methods and variable selection techniques

Electricity Theft Detection in Power Consumption Data Based on Adaptive Tuning Recurrent Neural Network

Contact Info

Product

Resources

About