Towards Data Reliability Based on Triple Redundancy and Online Outlier Detection

Abstract

Today, air quality monitoring is a global concern. The World Health Organization (WHO) defined standards for each pollutant and each member state is committed to monitoring them continuously and reliably to protect the population. This responsibility is delegated to air quality monitoring associations. To achieve the objectives of reliable, accurate, and continuous measurements, these associations rely on conventional measuring stations with demanding specifications to serve as scientific references and decision supports for the authorities. However, because of heavy investments and required qualified staff, there are few stations and the coverage is coarse for territories of several thousand km2. To circumvent this difficulty, measurement network architectures using Low-Cost Sensors (LCS) have been deployed. Low cost and requiring less qualification, This alternative technology to conventional measuring stations makes it possible to target local pollution that could not otherwise be detected. Although it is more accurate on the spatial dimension, this technology has several drawbacks, notably in terms of measurement repeatability and hardware quality. It is also subject to measurement drifts over time. To overcome these drawbacks, a resilient and reliable architecture based on LCS and triple redundancy has been proposed. The basic principle is based on the implementation of three smart sensors (SmS) using LCS measuring the same parameters on the same perimeter. These SmS communicate with an Aggregator that aggregates the data sent by SmS. The aggregator includes also detection and voting tasks allowing to compare, cross the data, detect faults of LCS online, and provide data that are ready for processing. In this paper, a pre-processing algorithm in four steps is presented. It identifies hardware faults from one or more LCS and reports outliers for verification by an expert. It is configurable and can identify failure behaviors (LCS or air quality). Finally, the proposed algorithm excludes the outliers data from faulty LCS and presents only reliable ones.