Convolutional neural network for simultaneous prediction of several soil properties using visible/near-infrared, mid-infrared, and their combined spectra

Ng, Wartini; Minasny, Budiman; Montazerolghaem, Maryam; Padarian, José; Ferguson, Richard; Bailey, Scarlett; McBratney, Alex B.

doi:10.1016/j.geoderma.2019.06.016

Cited by 281 publications

(164 citation statements)

References 38 publications

Supporting

Mentioning

156

Contrasting

Unclassified

Order By: Relevance

“…Another alternative, in the context of soil mapping, is to map the rules generated by the model to identify their spatial context or to map where important predictors were used (Bui et al, 2006). For CNN, by feeding simulated data to a trained model, Ng et al (2019) explored the most important wavelengths used when predicting multiple soil properties from soil spectral data using a sensitivity analysis. The logic behind their analysis is that modifying unimportant wavelengths should not affect the prediction.…”

Section: How Can We Increase Interpretability?mentioning

confidence: 99%

See 1 more Smart Citation

Machine learning and soil sciences: a review aided by machine learning tools

Padarian

Minasny

McBratney

2020

SOIL

Self Cite

236

View full text Add to dashboard Cite

Abstract. The application of machine learning (ML) techniques in various fields of science has increased rapidly, especially in the last 10 years. The increasing availability of soil data that can be efficiently acquired remotely and proximally, and freely available open-source algorithms, have led to an accelerated adoption of ML techniques to analyse soil data. Given the large number of publications, it is an impossible task to manually review all papers on the application of ML in soil science without narrowing down a narrative of ML application in a specific research question. This paper aims to provide a comprehensive review of the application of ML techniques in soil science aided by a ML algorithm (latent Dirichlet allocation) to find patterns in a large collection of text corpora. The objective is to gain insight into publications of ML applications in soil science and to discuss the research gaps in this topic. We found that (a) there is an increasing usage of ML methods in soil sciences, mostly concentrated in developed countries, (b) the reviewed publications can be grouped into 12 topics, namely remote sensing, soil organic carbon, water, contamination, methods (ensembles), erosion and parent material, methods (NN, neural networks, SVM, support vector machines), spectroscopy, modelling (classes), crops, physical, and modelling (continuous), and (c) advanced ML methods usually perform better than simpler approaches thanks to their capability to capture non-linear relationships. From these findings, we found research gaps, in particular, about the precautions that should be taken (parsimony) to avoid overfitting, and that the interpretability of the ML models is an important aspect to consider when applying advanced ML methods in order to improve our knowledge and understanding of soil. We foresee that a large number of studies will focus on the latter topic.

show abstract

Section: How Can We Increase Interpretability?mentioning

confidence: 99%

“…This analysis allows us to explore the most important wavelengths in a CNN model. Adapted fromNg et al (2019).…”

mentioning

confidence: 99%

Machine learning and soil sciences: a review aided by machine learning tools

Padarian

Minasny

McBratney

2020

SOIL

Self Cite

236

View full text Add to dashboard Cite

show abstract

“…Another alternative, in the context of soil mapping, is to map the rules generated by the model to identify their spatial context or to map where important predictors were used (Bui et al, 2006). For CNNs, by feeding simulated data to a trained model, Ng et al (2019) explored the most important wavelengths used when predicting multiple soil properties from soil spectral data using a sensitivity analysis. The logic behind their analysis is that modifying unimportant wavelengths should not affect the prediction.…”

Section: How Can We Increase Interpretability?mentioning

confidence: 99%

Machine learning and soil sciences: A review aided by machine learning tools

Padarian¹,

Minasny²,

McBratney³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Abstract. The application of machine learning (ML) techniques in various fields of science has increased rapidly, especially in the last ten years. The increasing availability of soil data that can be efficiently acquired remotely and proximally, and freely available open-source algorithms, have led to an accelerated adoption of ML techniques to analyse soil data. Given the large number of publications, it is an impossible task to manually review all papers on the application of ML in soil science without narrowing down a narrative of ML application in a specific research question. This paper aims to provide a comprehensive review of the application of ML techniques in soil science aided by a ML algorithm (Latent Dirichlet Allocation) to find patterns in a large collection of text corpus. The objective is to gain insight into publications of ML applications in soil science and to discuss the research gaps in this topic. We found that: a) there is an increasing usage of ML methods in soil sciences, mostly concentrated in developed countries, b) the reviewed publication can be grouped into 12 topics, namely remote sensing, soil organic carbon, water, contamination, methods (ensembles), erosion and parent material, methods (NN, SVM), spectroscopy, modelling (classes), crops, physical and modelling (continuous), c) advanced ML methods usually perform better than simpler approaches thanks to their capability to capture non-linear relationships. From these findings, we found research gaps, in particular: about the precautions that should be taken (parsimony) to avoid overfitting, and that the interpretability of the ML models is an important aspect to consider when applying advanced ML methods in order to improve our knowledge and understanding of soil. We foresee that a large number of studies will focus on the latter topic.

show abstract

“…In previous study (Ng et al, 2019), we calculated the sensitivity as a function of the variance of the model for each window of 183 spectra. Here we calculate the sensitivity based on the variance principle as an alternative approach: 184…”

Section: Sensitivity Analysis: Evaluating Important Wavelengths 175mentioning

confidence: 99%

“…Discussion started: 17 September 2019 c Author(s) 2019. CC BY 4.0 License.The CNN model utilized in this study is derived from our previous study(Ng et al, 2019), where the spectra data were fed 121 into the model as a one-dimensional data. The architecture of the CNN model is included inTable 2and Figure 2 .…”

mentioning

confidence: 99%

Estimation of effective calibration sample size using visible near infrared spectroscopy: deep learning vs machine learning

Minasny

Mendes

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

8The number of samples used in the calibration dataset affects the quality of the generated predictive models using visible, near 9 and shortwave infrared (VIS-NIR-SWIR) spectroscopy for soil attributes. Recently, convolutional neural network (CNN) is 10 regarded as a highly accurate model for predicting soil properties on a large database, however it has not been ascertained yet 11how large the sample size should be for CNN model to be effective. This paper aims at providing an estimate of how much 12 calibration samples are needed to improve the model performance of soil properties predictions with CNN. It is hypothesized 13 that the larger the amount of data, the more accurate is the CNN model. The performances of two commonly used machine 14 learning models (Partial least squares regression (PLSR) and Cubist) are compared against the CNN model. A VIS-NIR-SWIR 15 spectral library from Brazil containing 4251 unique sites, with averages of 2-3 samples per depth (a total of 12,044 samples), 16 was divided into calibration (3188 sites) and validation (1063 sites) sets.

show abstract

Convolutional neural network for simultaneous prediction of several soil properties using visible/near-infrared, mid-infrared, and their combined spectra

Cited by 281 publications

References 38 publications

Machine learning and soil sciences: a review aided by machine learning tools

Machine learning and soil sciences: a review aided by machine learning tools

Machine learning and soil sciences: A review aided by machine learning tools

Estimation of effective calibration sample size using visible near infrared spectroscopy: deep learning vs machine learning

Contact Info

Product

Resources

About