Validating data quality during wet weather monitoring of wastewater treatment plant influents

Abstract

Efficient monitoring of water systems and proper use of the collected data in further applications such as modelling, forecasting influent water quality and real-time control depends on careful data quality control. Given the size of the data sets produced nowadays in online water quality monitoring schemes, automated data validation is the only feasible option. In this paper, software tools for automatic data quality assessment with a practical orientation are presented. The developments from three organizations ranging from simple to more complex methods for automated data validation are described and evaluated for water quality measurements collected at the inlet of wastewater treatment plants, where probably the hardest measurement conditions are found. The objective of this collaborative effort is to come up with better tools and improved approaches for implementing a successful automatic data quality control procedure.

A Survey on Data Quality for Dependable Monitoring in Wireless Sensor Networks

“…One recent example [ 66 ] presents an algorithm to correct rough and missing information grounded on Kalman filtering to surpass the issue with querying faulty information and to enhance the exactness of data in a 1000-node WSN in a synthetic environment. Another example is presented in Reference [ 67 ], in the context of an aquatic monitoring application, in which Kalman filtering was used with forecasting algorithms to assess the quality of the monitoring data series.…”

A full‐scale operational digital twin for a water resource recovery facility—A case study of Eindhoven Water Resource Recovery Facility

“…Online sensors at a WRRF are usually prone to anomalies (e.g., noise, failure, drift, and bias), which can dramatically affect the quality and/or the performance of model simulations. Although some studies have investigated anomaly detection and gap‐filling of wastewater data (Alferes et al, 2015; De Mulder et al, 2018; Martin & Vanrolleghem, 2014), they do not focus on the real‐time functionalities that are required in DT applications. In addition, there should be an automated connection between the data processing pipeline, which is mostly implemented in data‐driven programming languages like R and Python, and the process model simulators, which are usually developed using commercial software like WEST (DHI A/S, Denmark), SUMO (Dynamita, France), GPS‐X (Hatch, Canada), SIMBA# (inCTRL, Canada), and BioWin (EnviroSim, Canada).…”