Transferability of a large mid‐infrared soil spectral library between two Fourier‐transform infrared spectrometers

Abstract

Large and publicly available soil spectral libraries, such as the USDA National Soil Survey Center-Kellogg Soil Survey Laboratory (NSSC-KSSL) mid-infrared (MIR) spectral library, are enormously valuable resources enabling laboratories around the world to make rapid low-cost estimates of a number of soil properties. A limitation to widespread sharing of soil spectral data is the need to ensure that spectra collected on a secondary spectrometer are compatible with the spectra in the primary or reference library. Various spectral preprocessing and calibration transfer techniques have been proposed to overcome this limitation. We tested the transferability of models developed using the USDA NSSC-KSSL MIR library to a secondary instrument. For the soil properties, total carbon (TC), pH, and clay content, we found that good performance (ratio of performance to deviation [RPD] = 4.9, 2.0, and 3.6, respectively) could be achieved on an independent test set with Savitzky-Golay smoothing and first derivative preprocessing of the secondary spectra using a memory-based learning chemometric approach. We tested three calibration transfer techniques (direct standardization [DS], piecewise direct standardization [PDS], and spectral space transformation [SST]) using different size transfer sets selected to be representative of the entire NSSC-KSSL library. Among the transfer methods, SST consistently outperformed DS and PDS with 50 transfer samples being an optimal number for transfer model development. For TC and pH, performance was improved using the SST transfer (RPD = 7.7 and 2.2, respectively) primarily through the elimination of bias. Calibration transfer could not improve predictions for clay. These findings suggest that calibration transfer may not always be necessary, but users should test to confirm this assumption using a small set of representative samples scanned at both laboratories.

Open Soil Spectral Library (OSSL): Building reproducible soil calibration models through open development and community engagement

“…2,016 of these soil samples were selected to represent the diversity of mineral soils found in the United States, and 90 samples were selected across Ghana, Kenya, and Nigeria. The 2,016 soil samples were obtained from USDA NRCS NSSC-KSSL Soil Archives and scanned by the Woodwell Climate Research Center and the University of Nebraska — Lincoln, which were made available with a CC-BY 4.0 license on Zenodo [32]. The number of unique samples correctly imported into the OSSL per original source and type of spectra is described in Table 1.…”

“…The same soil properties were evaluated using VisNIR, i.e., , , and . Lastly, the NIR Neospectra prediction models were tested with 90 soil samples from Ghana, Kenya, and Nigeria available through the public Neospectra database [32]. These samples were not used during model calibration but rather kept separated for external validation.…”

An Interlaboratory Comparison of Mid-Infrared Spectra Acquisition: Instruments and Procedures Matter

“…2) for proper evaluation of modeling results. Kennard-Stone deterministic sampling algorithm (Kennard and Stone, 1969) was run on SNV KSSL spectra to subset 50 samples, an optimal number for spectral standardization defined as per the previous analysis of Sanderman et al (2023). Before the subsetting, the KSSL SNV spectra were compressed by principal component analysis (PCA) to retain 99.99 % of the original variability.…”

“…Spectral space transformation is a relatively new method and was first described by Du et al (2011). Its adoption in soil spectroscopy is very limited, although some recent studies have indicated that SST outperforms other spectral standardization methods, i.e., direct standardization (DS) and piecewise direct standardization (PDS), especially when a small number of standard samples are available to be shared across laboratories or instruments (Pittaki-Chrysodonta et al, 2021;Sanderman et al, 2023). The method is based on the transformation of spectra of a secondary instrument onto the spectral space of the primary one.…”

“…Dangal and Sanderman (2020) found that spectral standardization was necessary to deliver unbiased predictions for a test set of different geographical origin despite reaching good-to-excellent results without standardization. In turn, Sanderman et al (2023) suggested that calibration transfer may not be necessary when they transferred a model from the USDA National Soil Survey Center Kellogg Soil Survey Laboratory (NSSC KSSL) FT-MIR spectral library to a localized spectral library collected with a different instrument model from the same manufacturer, with spectral preprocessing being enough to handle the spectral variability.…”