Reducing moisture effects on soil organic carbon content prediction in visible and near-infrared spectra with an external parameter othogonalization algorithm

Abstract

Field spectroscopy and other efficient hyperspectral techniques have been widely used to measure soil properties, including soil organic carbon (SOC) content. However, reflectance measurements based on field spectroscopy are quite sensitive to uncontrolled variations in surface soil conditions, such as moisture content; hence, such variations lead to drastically reduced prediction accuracy. The goals of this work are to (i) explore the moisture effect on soil spectra with different SOC levels, (ii) evaluate the selection of optimal parameter for external parameter othogonalization (EPO) in reducing moisture effect, and (iii) improve SOC prediction accuracy for semi-arid soils with various moisture levels by combing the EPO with machine learning method. Soil samples were collected from grassland regions of Inner Mongolia in North China. Rewetting laboratory experiments were conducted to make samples moisturized at five levels. Visible and near-infrared spectra (350–2500 nm) of soil samples rewetted were observed using a hand-held SVC HR-1024 spectroradiometer. Our results show that moisture influences the correlation between SOC content and soil reflectance spectra and that moisture has a greater impact on the spectra of samples with low SOC. An EPO algorithm can quantitatively extract information of the affected spectra from the spectra of moist soil samples by an optimal singular value. A SOC model that effectively couples EPO with random forest (RF) outperforms partial least-square regression (PLSR)-based models. The EPO–RF model generates better results with R2 of 0.86 and root-mean squared error (RMSE) of 3.82 g kg−1, whereas a PLSR model gives R2 of 0.79 and RMSE of 4.68 g kg−1.

Spectra transfer based learning for predicting and classifying soil texture with short-ranged Vis-NIRS sensor

“…Observations resided outside the convex hull of wet and PDS transferred spectra indicated different characteristics than samples that resided around the centroid surrounded by all convex hulls. This also meant that the feature information of those spectra was absent in wet and PDS transferred spectra, which was influenced by soil MC, a dominating agent to affect spectra information (Cao et al, 2020). In a single word, the air-dried soil and DS transferred spectra were almost equally distributed over the convex hull biplot, and thus these two types of spectra should reserve the most key spectral information.…”

“…This means that the use of the current Vis-NIRS for the analysis and classification of soil texture classes based on wet soil spectra is a complete failure. This is due to the significant influence of the MC has on the texture fraction, particularly on the clay content, as clay changes in volume (at different degrees according to the type of minerals) with soil MC, leading to different spectroscopy responses (absorption/reflectance of light) at different MC (Cao et al, 2020).…”

Comparing Laboratory and Satellite Hyperspectral Predictions of Soil Organic Carbon in Farmland

“…Proximal hyperspectral technology, particularly visible and near-infrared (vis-NIR) spectroscopy, has become a significant data source for DSM, providing a practical and economical method for rapid and accurate soil property estimation [5,9]. Although studies mostly occur under laboratory conditions due to the higher estimation accuracy of laboratory spectral data, discrete soil samples provide limited point-to-point data [10].…”

Construction and Evaluation of Prediction Model of Main Soil Nutrients Based on Spectral Information

“…It was revealed by a study that soil hyperspectral detection can be severely disrupted by soil moisture content in the water absorption bands (1400, 1900, and 2200 nm) [16], thus resulting in a poor applicability of the prediction model [17]. In this respect, external parameter orthogonalization (EPO) and machine learning can be combined to improve the accuracy of organic carbon prediction in semiarid soils with varying water contents [18]. The soil organic matter estimation model is established by using the hyperspectral data processed by EPO, which prevents the potential impact of soil moisture [17].…”