Evaluating calibration methods for predicting soil available nutrients using hyperspectral VNIR data

Abstract:Field spectroscopy has been suggested to be an efficient method for predicting soil properties using quantitative mathematical models in a rapid and non-destructive manner. Traditional multivariate regression algorithms usually regard the modeling of each soil property as a single task, which means only one response variable is considered as the output during modeling. Therefore, these algorithms are less suitable for the prediction of several key soil properties with low concentrations or unobvious spectral absorption signals. In the current study, we investigated the performance of a linear multi-task learning (LMTL) algorithm based on a regularized dirty model for modeling and predicting several key soil properties using field spectroscopy (350-2500 nm) as an integrated approach. We tested seven key soil properties including available nitrogen (N), phosphorus (P) and potassium (K), pH, water content (WC), organic matter (OM), and electrical conductivity (EC) in drylands. The model performances of LMTL models were compared with the commonly used single-task algorithm of the partial least squares regression (PLS-R). Our results show that the LMTL models outperformed the PLS-R models with the advantage of shared features; the ratio of performance to deviation (RPD) values in the validation set improved by 10.24%, 4.93%, 25.77%, 11.76%, 6.74%, 53.13%, and 3.15% for N, P, K, pH, WC, OM, and EC, respectively. The best prediction was obtained for OM with RPD = 2.29, indicating high accuracy (RPD > 2). The prediction results of N, P, WC, and pH were categorized as of moderate accuracy (1.4 < RPD < 2), while K and EC were categorized as of poor accuracy (RPD < 1.4). However, the explanatory power of the LMTL models was moderate due to fewer features being selected by the regularization algorithm of the LMTL approach, which should be further studied in the soil spectral analysis. Our results highlight the use of LMTL in field spectroscopy analysis that can improve the generalization performance of regression models for predicting soil properties.

show abstract

“…A good model should have both high RPD and SSR/SST [13]. Usually, the SSR/SST should be greater than 0.5 to ensure the model stability.…”

Section: Accuracy Comparisonmentioning

confidence: 99%

Linear Multi-Task Learning for Predicting Soil Properties Using Field Spectroscopy

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…Calibration method is the specified rule that calibration procedures followed, leading to the attainment of certain additional analytical goals (Kościelniak and Wieczorek ). It includes different combinations of preprocessing transformations and regression algorithms (Qi and others ). The preprocessing transfromations generally used in hyperspectral imaging are normalization (N), multiplicative scatter correction (MSC), standard normal variate (SNV), first derivative (1st D), and second derivative (2nd D) (Feng and others ).…”

Section: Methodsmentioning

confidence: 99%

“…1st D and 2nd D are applied to remove background noise, baseline drift and enhance small spectral features (Shen and others ). For regression algorithms (RAs), partial least squares regression (PLSR), least‐squares support vector machine (LS‐SVM), stepwise multiple linear regression (SMLR), backpropagation neural network (BPNN), and support vector machine (SVM) are commonly employed to interpret complex relationship between spectra and measured attributes (Feng and others ; Qi and others ). Among those RAs, PLSR is a useful and powerful multivariate data method to analyse data with numerous and strongly collinear variables in the independent variables (X) and dependent variables (Y) (Jia and others ; Qiao and others ; Wang and others ).…”

Section: Methodsmentioning

confidence: 99%

Hyperspectral Imaging in Tandem with R Statistics and Image Processing for Detection and Visualization of pH in Japanese Big Sausages Under Different Storage Conditions

Feng

Makino

Yoshimura

et al. 2017

Journal of Food Science

View full text Add to dashboard Cite

In this paper, hyperspectral imaging is for the first time used to detect pH in cooked sausages using R statistics, which provides another useful information for the researchers who do not have the access to Matlab. Eleven optimal wavelengths were successfully selected, which were used for simplifying the PLSR model established based on the full wavelengths. This simplified model achieved a high R (0.909) and a low root mean square error of prediction (0.035), which can be useful for the design of multispectral imaging systems.

show abstract

“…RMSE included the RMSE of calibration (RMSEC) and the RMSE of validation (RMSEP). According to Qi et al (Qi et al 2018), it is feasible to adopt three categories of criteria to assess model predictability: category I (RPD > 2.0) with excellent predictability; category II (1.4 < RPD < 2.0) with moderate predictability; and category III (RPD < 1.4) with poor predictability.…”

Section: Model Evaluation and Comparisonmentioning

confidence: 99%

Untitled

Supplemental Information 7: r² Maps of 2D Spectral Indices Based on SDA

View full text Add to dashboard Cite

Soil moisture content (SMC) is an important factor that affects agricultural development in arid regions. Compared with the spaceborne remote sensing system, the unmanned aerial vehicle (UAV) has been widely used because of its stronger controllability and higher resolution. It also provides a more convenient method for monitoring SMC than normal measurement methods that includes field sampling and oven-drying techniques. However, research based on UAV hyperspectral data has not yet formed a standard procedure in arid regions. Therefore, a universal processing scheme is required. We hypothesized that combining pretreatments of UAV hyperspectral imagery under optimal indices and a set of field observations within a machine learning framework will yield a highly accurate estimate of SMC. Optimal 2D spectral indices act as indispensable variables and allow us to characterize a model's SMC performance and spatial distribution. For this purpose, we used hyperspectral imagery and a total of 70 topsoil samples (0-10 cm) from the farmland (2.5×10 4 m 2) of Fukang City, Xinjiang Uygur Autonomous Region, China. The random forest (RF) method and extreme learning machine (ELM) were used to estimate the SMC using six methods of pretreatments combined with four optimal spectral indices. The validation accuracy of the estimated method clearly increased compared with that of linear models. The combination of pretreatments and indices by our assessment effectively eliminated the interference and the noises. Comparing two machine learning algorithms showed that the RF models were superior to the ELM models, and the best model was PIR (R 2 val = 0.907, RMSEP = 1.477 and RPD = 3.396). The SMC map predicted via the best scheme was highly similar to the SMC map measured. We conclude that combining preprocessed spectral indices and machine learning algorithms allows estimation of SMC with high accuracy (R 2 val = 0.907) via UAV hyperspectral imagery on a regional scale. Ultimately, our program might improve management and conservation strategies for agroecosystem systems in arid regions.

show abstract

Evaluating calibration methods for predicting soil available nutrients using hyperspectral VNIR data

Cited by 51 publications

References 49 publications

Linear Multi-Task Learning for Predicting Soil Properties Using Field Spectroscopy

Linear Multi-Task Learning for Predicting Soil Properties Using Field Spectroscopy

Hyperspectral Imaging in Tandem with R Statistics and Image Processing for Detection and Visualization of pH in Japanese Big Sausages Under Different Storage Conditions

Untitled

Contact Info

Product

Resources

About