Prediction of E. coli Concentrations in Agricultural Pond Waters: Application and Comparison of Machine Learning Algorithms

Abstract

The microbial quality of irrigation water is an important issue as the use of contaminated waters has been linked to several foodborne outbreaks. To expedite microbial water quality determinations, many researchers estimate concentrations of the microbial contamination indicator Escherichia coli (E. coli) from the concentrations of physiochemical water quality parameters. However, these relationships are often non-linear and exhibit changes above or below certain threshold values. Machine learning (ML) algorithms have been shown to make accurate predictions in datasets with complex relationships. The purpose of this work was to evaluate several ML models for the prediction of E. coli in agricultural pond waters. Two ponds in Maryland were monitored from 2016 to 2018 during the irrigation season. E. coli concentrations along with 12 other water quality parameters were measured in water samples. The resulting datasets were used to predict E. coli using stochastic gradient boosting (SGB) machines, random forest (RF), support vector machines (SVM), and k-nearest neighbor (kNN) algorithms. The RF model provided the lowest RMSE value for predicted E. coli concentrations in both ponds in individual years and over consecutive years in almost all cases. For individual years, the RMSE of the predicted E. coli concentrations (log10 CFU 100 ml−1) ranged from 0.244 to 0.346 and 0.304 to 0.418 for Pond 1 and 2, respectively. For the 3-year datasets, these values were 0.334 and 0.381 for Pond 1 and 2, respectively. In most cases there was no significant difference (P > 0.05) between the RMSE of RF and other ML models when these RMSE were treated as statistics derived from 10-fold cross-validation performed with five repeats. Important E. coli predictors were turbidity, dissolved organic matter content, specific conductance, chlorophyll concentration, and temperature. Model predictive performance did not significantly differ when 5 predictors were used vs. 8 or 12, indicating that more tedious and costly measurements provide no substantial improvement in the predictive accuracy of the evaluated algorithms.

Oversampling methods for machine learning model data training to improve model capabilities to predict the presence of Escherichia coli MG1655 in spinach wash water

“…in the validation wash water data. In some non-linear based prediction models like SVM and RF, preprocessing methods as described in this study have very few positive effects on model performance and in some cases, such as in this study, can have negative impacts on the model performance (Stocker et al, 2022).…”

“…However, the preprocessing of the data had a significantly negative impact ( p < 0.05) on the ability for SVM and RF to detect the presence of E. coli MG1655 in the validation wash water data. In some non‐linear based prediction models like SVM and RF, preprocessing methods as described in this study have very few positive effects on model performance and in some cases, such as in this study, can have negative impacts on the model performance (Stocker et al., 2022).…”

“…Stocker et al. (2022) also found that conductivity consistently ranked in the top‐two most important predictive features when using water samples to detect E. coli .…”

“…Therefore, unlike the study conducted by Weller, Love, et al (2020), results from this study suggest that conductivity (rather than turbidity) is the most important physiochemi-cal feature of the wash water in improving model accuracy for detecting the presence of E. coli MG1655. Stocker et al (2022) also found that conductivity consistently ranked in the top-two most important predictive features when using water samples to detect E. coli.…”

“…There was a significant difference (p < 0.05) between all AUC values of the LR models. These differences indicate that there is a fundamental difference between the way that the oversampling methods improve the model accuracy for the RF, SVM, and LR models (Stocker et al, 2022). Model performance metrics demonstrate that there is no significant difference between the model performance in SVM or RF (p < 0.05) when using either the SMOTE method or the ADASYN method.…”