Predictive performance of mobile vis-near infrared spectroscopy for key soil properties at different geographical scales by using spiking and data mining techniques

Nawar, Said; Mouazen, Abdul Mounem

doi:10.1016/j.catena.2016.12.014

Cited by 120 publications

(60 citation statements)

References 52 publications

(88 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…When it comes to the validation procedure, cross validation [117,123] is the most commonly-used method due to the small sample size of the dataset, though one should keep in mind that there is the risk of overfitting the model; therefore, testing its performance on independent datasets usually provides less accuracy [46].…”

Section: Discussionmentioning

confidence: 99%

“…To address the lack of adequate local data for SOC and to better exploit existing SSls, the spiking technique is frequently evaluated. It is mostly used to augment an existing SSL with local spectra to improve the models accuracy [122][123][124]. Nawar et al [125] showed that by using spiking for model calibration, the sample selection method and the number of selected samples also affects the model's accuracy.…”

Section: Spiking Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

From Laboratory to Proximal Sensing Spectroscopy for Soil Organic Carbon Estimation—A Review

et al. 2020

View full text Add to dashboard Cite

Rapid and cost-effective soil properties estimations are considered imperative for the monitoring and recording of agricultural soil condition for the implementation of site-specific management practices. Conventional laboratory measurements are costly and time-consuming, and, therefore, cannot be considered appropriate for large datasets. This article reviews laboratory and proximal sensing spectroscopy in the visible and near infrared (VNIR)–short wave infrared (SWIR) wavelength region for soil organic carbon and soil organic matter estimation as an alternative to analytical chemistry measurements. The aim of this work is to report the progress made in the last decade on data preprocessing, calibration approaches, and system configurations used for VNIR-SWIR spectroscopy of soil organic carbon and soil organic matter estimation. We present and compare the results of over fifty selective studies and discuss the factors that affect the accuracy of spectroscopic measurements for both laboratory and in situ applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Spiking Techniquesmentioning

confidence: 99%

From Laboratory to Proximal Sensing Spectroscopy for Soil Organic Carbon Estimation—A Review

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The concentrations of soil Fe oxides and clay minerals should also be considered as outputs to enhance the learning ability of the LMTL models. Previous studies have argued that a nonlinear correlation exists between soil properties and spectral features (e.g., [16][17][18][19][20]). A future study should be conducted to apply nonlinear multi-task learning algorithms, such as a deep neural network [78,79], focusing on optimizing these algorithms to improve both the prediction accuracy and the explanatory power.…”

Section: Next Stepsmentioning

confidence: 99%

Linear Multi-Task Learning for Predicting Soil Properties Using Field Spectroscopy

et al. 2017

View full text Add to dashboard Cite

Abstract:Field spectroscopy has been suggested to be an efficient method for predicting soil properties using quantitative mathematical models in a rapid and non-destructive manner. Traditional multivariate regression algorithms usually regard the modeling of each soil property as a single task, which means only one response variable is considered as the output during modeling. Therefore, these algorithms are less suitable for the prediction of several key soil properties with low concentrations or unobvious spectral absorption signals. In the current study, we investigated the performance of a linear multi-task learning (LMTL) algorithm based on a regularized dirty model for modeling and predicting several key soil properties using field spectroscopy (350-2500 nm) as an integrated approach. We tested seven key soil properties including available nitrogen (N), phosphorus (P) and potassium (K), pH, water content (WC), organic matter (OM), and electrical conductivity (EC) in drylands. The model performances of LMTL models were compared with the commonly used single-task algorithm of the partial least squares regression (PLS-R). Our results show that the LMTL models outperformed the PLS-R models with the advantage of shared features; the ratio of performance to deviation (RPD) values in the validation set improved by 10.24%, 4.93%, 25.77%, 11.76%, 6.74%, 53.13%, and 3.15% for N, P, K, pH, WC, OM, and EC, respectively. The best prediction was obtained for OM with RPD = 2.29, indicating high accuracy (RPD > 2). The prediction results of N, P, WC, and pH were categorized as of moderate accuracy (1.4 < RPD < 2), while K and EC were categorized as of poor accuracy (RPD < 1.4). However, the explanatory power of the LMTL models was moderate due to fewer features being selected by the regularization algorithm of the LMTL approach, which should be further studied in the soil spectral analysis. Our results highlight the use of LMTL in field spectroscopy analysis that can improve the generalization performance of regression models for predicting soil properties.

show abstract

“…Models derived with spectra collected in laboratory are used for on-line prediction of soil properties, assuming that laboratory and on-line measured spectra are mutually substitutable and ignoring spectral discrepancies between them [12,21,22]. However, many external factors such as variation of soil-to-sensor distance and angle, noise due to mechanical vibrations, presence of soil debris, and ambient light can have great influences on overall spectral quality [21] during on-line measurement, hence, introduce considerable discrepancies between one-line and laboratory collected spectrum of the same soil.…”

Section: Introductionmentioning

confidence: 99%

Estimation of Secondary Soil Properties by Fusion of Laboratory and On-Line Measured Vis–NIR Spectra

2019

Self Cite

View full text Add to dashboard Cite

Visible and near infrared (vis–NIR) diffuse reflectance spectroscopy has made invaluable contributions to the accurate estimation of soil properties having direct and indirect spectral responses in NIR spectroscopy with measurements made in laboratory, in situ or using on-line (while the sensor is moving) platforms. Measurement accuracies vary with measurement type, for example, accuracy is higher for laboratory than on-line modes. On-line measurement accuracy deteriorates further for secondary (having indirect spectral response) soil properties. Therefore, the aim of this study is to improve on-line measurement accuracy of secondary properties by fusion of laboratory and on-line scanned spectra. Six arable fields were scanned using an on-line sensing platform coupled with a vis–NIR spectrophotometer (CompactSpec by Tec5 Technology for spectroscopy, Germany), with a spectral range of 305–1700 nm. A total of 138 soil samples were collected and used to develop five calibration models: (i) standard, using 100 laboratory scanned samples; (ii) hybrid-1, using 75 laboratory and 25 on-line samples; (iii) hybrid-2, using 50 laboratory and 50 on-line samples; (iv) hybrid-3, using 25 laboratory and 75 on-line samples, and (v) real-time using 100 on-line samples. Partial least squares regression (PLSR) models were developed for soil pH, available potassium (K), magnesium (Mg), calcium (Ca), and sodium (Na) and quality of models were validated using an independent prediction dataset (38 samples). Validation results showed that the standard models with laboratory scanned spectra provided poor to moderate accuracy for on-line prediction, and the hybrid-3 and real-time models provided the best prediction results, although hybrid-2 model with 50% on-line spectra provided equally good results for all properties except for pH and Na. These results suggest that either the real-time model with exclusively on-line spectra or the hybrid model with fusion up to 50% (except for pH and Na) and 75% on-line scanned spectra allows significant improvement of on-line prediction accuracy for secondary soil properties using vis–NIR spectroscopy.

show abstract

Predictive performance of mobile vis-near infrared spectroscopy for key soil properties at different geographical scales by using spiking and data mining techniques

Cited by 120 publications

References 52 publications

From Laboratory to Proximal Sensing Spectroscopy for Soil Organic Carbon Estimation—A Review

From Laboratory to Proximal Sensing Spectroscopy for Soil Organic Carbon Estimation—A Review

Linear Multi-Task Learning for Predicting Soil Properties Using Field Spectroscopy

Estimation of Secondary Soil Properties by Fusion of Laboratory and On-Line Measured Vis–NIR Spectra

Contact Info

Product

Resources

About