Optimization of spectral pre‐processing for estimating soil condition on small farms

Abstract

The concepts of soil security (especially relating to soil condition) provide a useful framework in building spectral libraries. Spectral libraries can be used with the purpose of assessing soil condition by measuring soil organic carbon (SOC) or increasing productivity through soil nutrient management. A spectral library was generated by measuring SOC and nutrients (nitrogen, phosphorous and potassium) and spectral reflectance data over the visible to near-infrared range (350-2,500 nm) in soil samples collected from four production systems in Papua New Guinea (PNG).The spectral library was analysed using SpecOptim, a software tool developed at the James Hutton Institute to explore spectral pre-processing and calibration options.From 192 model combinations of model, the best one was identified for each study area. Different combinations of data were also explored (e.g. by farm or all together).We believe that at the local-scale, soil carbon and nitrogen variability can be captured; however, the spectrally inactive properties such as phosphorous and potassium need to have a higher variability and therefore pooling is required in order to predict properties chemometrically. The SpecOptim software is a useful tool where analysis of spectral data can be difficult to determine. Specifically, it helped improve the accuracy of predictions by 2% for C and N (except for East New Britain site) compared with previously used pre-processing techniques and calibration models while automating identification of the optimal pre-processing approach. We believe that we have developed research-based evidence for using spectral libraries to fit with the soil priority areas of PNG.

Dataset variability and carbonate concentration influence the performance of local visible-near infrared spectral models

“…2) illustrates this finding by the different grouping of individual field datasets due to different pre-processing. This leads to the conclusion that studies that did not optimize the pre-processing scheme for every soil property separately did eventually not make full use of the spectroscopy potential (see also examples in Table 3), which has been shown by other studies as well (Alomar et al, 2021;Rodriguez-Febereiro et al, 2022;Singh et al, 2022). Nevertheless, the property-specific optimization of spectral pre-processing is a tedious process and constrains the fast and cost-effective application of vis-NIR spectroscopy, but some progress has been made simultaneously with our study by Mishra et al (2022).…”

“…It has been shown that local models improve with increasing numbers of calibration samples and that a sample size of at least 50 provides accurate prediction models (Kuang and Mouazen, 2012). Some studies thus combined multiple target sites and develop a general model by combining all the local datasets to reach a larger sample size and potentially better model performance (Kuang and Mouazen, 2011;Singh et al, 2022). In these studies, the general model showed an intermediate performance, and the general prediction error was between the best and the poorest performing local model.…”

Best performances of visible–near-infrared models in soils with little carbonate – a field study in Switzerland

“…2) illustrates this finding by the different groupings of individual field datasets due to different pre-processing. This leads to the conclusion that studies that did not optimize the preprocessing scheme for every soil property separately did eventually not make full use of the spectroscopy, which has been shown by other studies as well (Alomar et al, 2021;Rodriguez-Febereiro et al, 2022;Singh et al, 2022). Nevertheless, the property-specific optimization of spectral preprocessing is a tedious process and constrains the fast and cost-effective application of vis-NIR spectroscopy, but some progress has recently been made by Mishra et al (2022).…”

“…Kuang and Mouazen (2012) showed that local models improve with an increasing number of calibration samples and that a sample size of at least 50 provides accurate prediction models. Some studies thus combined multiple target sites and developed a general model by combining all the local datasets to reach a larger sample size and potentially better model performance (Kuang and Mouazen, 2011;Singh et al, 2022). In these studies, the general model showed an intermediate performance, and the general prediction error was between the best-and the poorest-performing local model.…”

Developing agriculture resilience in sub‐Saharan Africa

“…In the era of information, big data and relevant technology have the great potential of facilitating better understanding of agricultural available data. Remote sensing combined with laboratory analyses of samples could generate digital soil maps, building the farmland information system that will provide ready and efficient data‐driven planting strategy for smallholders (Piikki et al, 2020; Singh et al, 2020). Using the data to form soil indicators (other than merely judging by colour) or quick testing toolkits will make the evaluation of soil quality more precise and convenient (Mihoub et al, 2021).…”