Accurate and Precise Prediction of Soil Properties from a Large Mid-Infrared Spectral Library

Abstract

Diffuse reflectance spectroscopy (DRS) is emerging as a rapid and cost-effective alternative to routine laboratory analysis for many soil properties. However, it has primarily been applied in project-specific contexts. Here, we provide an assessment of DRS spectroscopy at the scale of the continental United States by utilizing the large (n > 50,000) USDA National Soil Survey Center mid-infrared spectral library and associated soil characterization database. We tested and optimized several advanced statistical approaches for providing routine predictions of numerous soil properties relevant to studying carbon cycling. On independent validation sets, the machine learning algorithms Cubist and memory-based learner (MBL) both outperformed random forest (RF) and partial least squares regressions (PLSR) and produced excellent overall models with a mean R2 of 0.92 (mean ratio of performance to deviation = 6.5) across all 10 soil properties. We found that the use of root-mean-square error (RMSE) was misleading for understanding the actual uncertainty about any particular prediction; therefore, we developed routines to assess the prediction uncertainty for all models except Cubist. The MBL models produced much more precise predictions compared with global PLSR and RF. Finally, we present several techniques that can be used to flag predictions of new samples that may not be reliable because their spectra fall outside of the calibration set.

Mid‐infrared spectroscopy for prediction of soil health indicators in the United States

“…In this work, we followed the same model development and testing procedures detailed in Dangal et al. ().…”

“…Each soil sample was scanned in quadruplicate and the spectra were averaged to provide a representative spectrum for each soil sample. These spectra span samples from all 50 states in the USA and the library has good representation of all master horizons and all soil orders except Oxisols (Dangal et al, 2019). Table 1 contains a description of all of the soil health properties that were tested in this study.…”

“…The major difference between PLSR and MBL modeling approaches is that with PLSR a single calibration model is developed using the training data while MBL can be thought of as a "just-in-time" approach where a new model using only the most representative samples in the training set is developed for each new sample to be predicted (Ramirez-Lopez et al, 2013). In this work, we followed the same model development and testing procedures detailed in Dangal et al (2019).…”

“…While this methodology is an excellent way of increasing sampling density without greatly increasing analytical costs, the models may not be broadly applicable. Several research groups have been working to build large national and even international spectral libraries and associated databases with the intention of turning diffuse reflectance spectroscopy into a routine production tool (Baldock, Hawke, Sanderman, & Macdonald, 2014;Dangal, Sanderman, Wills, & Ramirez-Lopez, 2019;Nocita et al, 2014;Terhoeven-Urselmans, Vagen, Spaargaren, & Shepherd, 2010;Viscarra Rossel et al, 2016;Wijewardane, Ge, Wills, & Loecke, 2016, 2018.…”

“…Here, we expand on the work introduced in Dangal et al (2019) utilizing the USDA National Soil Survey Center -Kellogg Soil Survey Laboratory (NSSC-KSSL) MIR spectral library and associated Soil Characterization Database (SCD) to determine which soil health indicators can be reasonably predicted using this large national database.…”