The Relocation Problem of Field Calibrated Low-Cost Sensor Systems in Air Quality Monitoring: A Sampling Bias

Abstract

This publication revises the deteriorated performance of field calibrated low-cost sensor systems after spatial and temporal relocation, which is often reported for air quality monitoring devices that use machine learning models as part of their software to compensate for cross-sensitivities or interferences with environmental parameters. The cause of this relocation problem and its relationship to the chosen algorithm is elucidated using published experimental data in combination with techniques from data science. Thus, the origin is traced back to insufficient sampling of data that is used for calibration followed by the incorporation of bias into models. Biases often stem from non-representative data and are a common problem in machine learning, and more generally in artificial intelligence, and as such a rising concern. Finally, bias is believed to be partly reducible in this specific application by using balanced data sets generated in well-controlled laboratory experiments, although not trivial due to the need for infrastructure and professional competence.

Relevance of Drift Components and Unit-to-Unit Variability in the Predictive Maintenance of Low-Cost Electrochemical Sensor Systems in Air Quality Monitoring

“…Such “anomalous” conditions include higher or lower values of either pollutant(s) but also different combinations compared to the ones in during the calibration phase. Thus, the measurement results can be completely unreliable during these periods—for instance, if the calibration function is extrapolated (e.g., random forests cannot extrapolate at all) [ 24 ]. Something that is not necessarily captured is the rotation of the probability distribution (i.e., changing entries in the covariance matrix) of the current atmosphere due to varying relationships between the pollutants [ 24 ].…”

“…Thus, the measurement results can be completely unreliable during these periods—for instance, if the calibration function is extrapolated (e.g., random forests cannot extrapolate at all) [ 24 ]. Something that is not necessarily captured is the rotation of the probability distribution (i.e., changing entries in the covariance matrix) of the current atmosphere due to varying relationships between the pollutants [ 24 ]. Unfortunately, devices 2 and 3 have more missing information compared to device 1, but the aforementioned trends appear to be the same ( Figure S2 ).…”

“…The analysis shows that, during operation, the gas levels are either higher (CO, NO 2 ) or lower (O 3 ), which then causes the increase in anomalies in Figure 2 . This concept drift is a consequence of field calibration [ 24 ], as more combustion for heating in the winter leads to higher levels of CO and NO 2 , while less radiation leads to a decrease in O 3 levels. Nonetheless, these spikes in the anomaly score can be prevented by extending the calibration range in a laboratory setting.…”

“…The measurement period can be considered comparably short, so a longer study might be needed. For this reason, the data set from De Vito et al [ 6 ], which has been characterized in an earlier publication [ 24 ], was also tested under the same procedure (two weeks of field calibration with sensors for CO, NO 2 , and O 3 from a different manufacturer); the results are depicted in Figure S3 . Note that the values for ν had to be increased to reach “convergence”.…”

“…Newer studies have shown that these models do not generalize well due to strong correlations within the field data, and the devices suffer from spatial and temporal relocation problems [ 24 , 25 ]. The underlying cause is the difference in probability distributions of the factors across different locations and times, known as concept drift.…”