Measuring the effectiveness of performance-based training

Abstract

This article describes a statistical analysis of small water systems' turbidity data within the framework of a logic model for the USEPA's Performance-Based Training (PBT) program. The logic model shows the theoretical linkages between optimization training for small system operators; operator application of optimization techniques; improvements in plant filtration performance; and public health protection. The analysis comprised two phases. For the first phase, the authors used Bayesian analysis of turbidity data to test the statistical significance of changes in finished water quality resulting from training for small water system operators. For the second phase, the authors estimated the potential health benefits resulting from measured improvements in filtration performance. Considering only the improved removal of the pathogen Cryptosporidium, the expected annual health benefit of PBT is about ten fewer cases of infection per thousand persons served (within a 95% credible interval 0 to 18 fewer infections), though there may be benefits associated with the removal of other pathogens. The article also describes factors contributing to uncertainty in the estimated potential health benefits. The proposed two-phase approach supports the USEPA's development of drinking water program indicators which are meaningful, measurable, broadly applicable and change-sensitive.

Recent Research Every Utility Manager Needs to Know About

“…Findings from this study show that the monetized benefits provided by reduced cases of gastroenteric disease exceeded the required investments by a factor of between 3 and 10, depending on how the health benefits were monetized. Bowman et al (2009) offered further evidence that additional investments in small systems can yield results. They reported that a performance‐based training system for small system operators significantly improved finished water turbidity and estimated that such training efforts can reduce the incidence of cryptosporidiosis by 10 cases per 1,000 customers.…”

Methods, Protocols, Guidance and Standards for Performance Evaluation for Point-of-Use Water Treatment Technologies: History, Current Status, Future Needs and Directions

“…Reduction of turbidity during drinking water treatment processes has been shown to be significantly correlated with removal of Giardia cysts and Cryptosporidium oocysts in both pilot [245] and full-scale [246] treatment facilities. Efforts to improve the quality of drinking water by more effective filtration through regulation [247] and through operator training [248] were predicated on turbidity reduction and associated removal of protozoan cysts and oocysts. Periodic measurements of the turbidity of both challenge water and product water during testing of a POU/HWT device may provide useful indications of the performance of the system.…”