Estimation of effective calibration sample size using visible near infrared spectroscopy: deep learning vs machine learning

Ng, Wartini; Minasny, Budiman; Mendes, Wanderson de Sousa; Demattê, José A. M.

doi:10.5194/soil-2019-48

Cited by 12 publications

(14 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most accurate model was the cubist, and that is the most applied model in the literature for soil predictions using Vis-NIR spectroscopy (Rossel et al, 2016;Demattê et al, 2019). As proved by Ng et al (2019), soil spectral libraries with up to 2000 observations fit better the models using cubist and over 2000 observations it suggests to use a convolutional neural network. The performance overall results for pH(H 2 O), sand, clay, and SOC were in according to those ranges reported by Soriano-Disla et al (2014), Rossel et al (2016), andDemattê et al (2019).…”

Section: Discussionmentioning

confidence: 90%

Soil spectral library of Piauí State using machine learning for laboratory analysis in Northeastern Brazil

Mendes¹,

Boechat²,

Gualberto³

et al. 2021

Revista Brasileira De Ciência Do Solo

Self Cite

View full text Add to dashboard Cite

Soil chemical and physical analyses are the major sources of data for agriculture. However, traditional soil analyses are time-consuming, not cost-efficient, and not environmentally friendly. An alternative to traditional soil analyses is soil spectroscopy. This technique is a low-cost and quick analytical method, which can be implemented in a laboratory and/or in-situ. Nevertheless, some spectrometers are expensive and do not contemplate the entire spectrum. Despite this limitation, the main objective of the study was to create a soil spectral library of the Piauí State using only the 1000-2500 nm range. In this sense, it was evaluated and standardized the soil spectral library by accessing the combination of smoothing, standard normal variate, continuum removal, and Savitzky-Golay derivative spectral preprocessing procedures with partial least squares, random forest, and cubist machine learning algorithms. It was collected 262 geo-referenced soil samples at the layer of 0.00-0.20 m across the entire Piauí State, representing most of its soil variability. The soil properties evaluated were pH(H 2 O), sand, clay, and soil organic carbon (SOC) contents. This study demonstrated that the Standard Normal Variate was one of the most promising preprocessing procedures to improve model predictions for pH(H 2 O), sand, and clay. For SOC and pH, the best overall results were without preprocessing the soil spectra. Moreover, the cubist model was the most accurate in predicting soil properties. Finally, our study showed evidence of the potential and feasibility of using this soil spectral library to estimate soil properties such as pH(H 2 O), sand, clay, and SOC.

show abstract

Section: Discussionmentioning

confidence: 90%

Soil spectral library of Piauí State using machine learning for laboratory analysis in Northeastern Brazil

Mendes¹,

Boechat²,

Gualberto³

et al. 2021

Revista Brasileira De Ciência Do Solo

Self Cite

View full text Add to dashboard Cite

show abstract

“…Nevertheless, our results only from the bare soils showed that 1D-CNN and LSTM had comparable accuracy with PLSR, illustrating that deep learning models did not have the advantage in dealing with a smaller dataset. Ng, Minasny, and Mendes, et al (2019) have shown that the PLSR performed better than the CNN when the training sample size was relatively small. However, when the sample size was more than 2,000, the CNN started to outperform the PLSR and cubist models, and its performance was still increasing with the increase of the sample size.…”

Section: Discussionmentioning

confidence: 99%

“…Padarian, Minasny, and McBratney (2019) have shown that multi-task convolutional neural network (CNN) performed significantly better than PLSR and cubist regression to predict soil properties from raw soil spectra. Ng, Minasny, Mendes, and Demattê (2019) trained a one-dimensional and a two-dimensional T A B L E 1 Summary of soil types, numbers of each soil type, and the range of organic matter…”

Section: Core Ideasmentioning

confidence: 99%

Predicting soil moisture content over partially vegetation covered surfaces from hyperspectral data with deep learning

Zhang

Liu

et al. 2021

Soil Science Soc of Amer J

View full text Add to dashboard Cite

Previous studies for retrieving soil moisture content (SMC) from visible and nearinfrared hyperspectral data over vegetation-covered surfaces using spectral unmixing, non-negative matrix factorization, and albedo/vegetation coverage in trapezoid spaces have required mass spectral preprocessing and offered only limited improvements in prediction accuracy. Recently, deep learning has triggered some improvements in soil properties prediction from hyperspectral data because of its automatic feature extraction and high accuracy. In this study, hyperspectral data in a simulation experiment with different vegetation coverages, SMCs, and soil types were acquired. Deep learning models, one-dimensional convolutional neural network (1D-CNN), and long short-term memory network (LSTM) are proposed to predict SMC. The results showed that two deep learning models achieved excellent predictions (residual prediction deviation [RPD] > 2.5) using the unpreprocessed mixed spectra and partial least squares regression (PLSR) had a good prediction (RPD = 1.88). The 1D-CNN (R 2 p = .91) and LSTM (R 2 p = .90) significantly outperformed PLSR (R 2 p = .72), which demonstrated that deep learning could improve SMC prediction over partially vegetation-covered surfaces. However, when only using bare soil spectra, the prediction accuracy was commensurate, whether through the 1D-CNN, LSTM, or PLSR models; additionally, 1D-CNN and LSTM had better performance on all mixed spectra than bare soil spectra. These results indicated that deep learning had no advantage on smaller datasets. We also found that SMC prediction with deep learning was affected by vegetation coverage and soil type but was still very good. The 1D-CNN and LSTM are effective models for predicting SMC with large hyperspectral datasets acquired from complex soil surface conditions.

show abstract

“…To determine which spectral wavelengths played important roles in the prediction of soil properties and vegetation coverage in the 1DCNN model, a sensitivity analysis based on the variance principle was implemented according to [40]. The results are shown in Figure 8.…”

Section: Determination Of Important Wavelengths Of 1dcnnmentioning

confidence: 99%

The Simultaneous Prediction of Soil Properties and Vegetation Coverage from Vis-NIR Hyperspectral Data with a One-Dimensional Convolutional Neural Network: A Laboratory Simulation Study

et al. 2022

View full text Add to dashboard Cite

Remote sensing of land surface mostly obtains a mixture of spectral information of soil and vegetation. It is thus of great value if soil and vegetation information can be acquired simultaneously from one model. In this study, we designed a laboratory experiment to simulate land surface compositions, including various soil types with varying soil moisture and vegetation coverage. A model of a one-dimensional convolutional neural network (1DCNN) was established to simultaneously estimate soil properties (organic matter, soil moisture, clay, and sand) and vegetation coverage based on the hyperspectral data measured in the experiment. The results showed that the model achieved excellent predictions for soil properties (R2 = 0.88–0.91, RPIQ = 4.01–5.78) and vegetation coverage (R2 = 0.95, RPIQ = 7.75). Compared with the partial least squares regression (PLSR), the prediction accuracy of 1DCNN improved 42.20%, 45.82%, 43.32%, and 36.46% in terms of the root-mean-squared error (RMSE) for predicting soil organic matter, sand, clay, and soil moisture, respectively. The improvement might be caused by the fact that the spectral preprocessing and spectral features useful for predicting soil properties were successfully identified in the 1DCNN model. For the prediction of vegetation coverage, although the prediction accuracy by 1DCNN was excellent, its performance (R2 = 0.95, RPIQ = 7.75, RMSE = 3.92%) was lower than the PLSR model (R2 = 0.98, RPIQ = 12.57, RMSE = 2.41%). These results indicate that 1DCNN can simultaneously predict soil properties and vegetation coverage. However, the factors such as surface roughness and vegetation type that could affect the prediction accuracy should be investigated in the future.

show abstract

Estimation of effective calibration sample size using visible near infrared spectroscopy: deep learning vs machine learning

Cited by 12 publications

References 9 publications

Soil spectral library of Piauí State using machine learning for laboratory analysis in Northeastern Brazil

Soil spectral library of Piauí State using machine learning for laboratory analysis in Northeastern Brazil

Predicting soil moisture content over partially vegetation covered surfaces from hyperspectral data with deep learning

The Simultaneous Prediction of Soil Properties and Vegetation Coverage from Vis-NIR Hyperspectral Data with a One-Dimensional Convolutional Neural Network: A Laboratory Simulation Study

Contact Info

Product

Resources

About