The use of hierarchical clustering for the design of optimized monitoring networks

Abstract

Abstract. Associativity analysis is a powerful tool to deal with large-scale datasets by clustering the data on the basis of (dis)similarity and can be used to assess the efficacy and design of air quality monitoring networks. We describe here our use of Kolmogorov–Zurbenko filtering and hierarchical clustering of NO2 and SO2 passive and continuous monitoring data to analyse and optimize air quality networks for these species in the province of Alberta, Canada. The methodology applied in this study assesses dissimilarity between monitoring station time series based on two metrics: 1−R, R being the Pearson correlation coefficient, and the Euclidean distance; we find that both should be used in evaluating monitoring site similarity. We have combined the analytic power of hierarchical clustering with the spatial information provided by deterministic air quality model results, using the gridded time series of model output as potential station locations, as a proxy for assessing monitoring network design and for network optimization. We demonstrate that clustering results depend on the air contaminant analysed, reflecting the difference in the respective emission sources of SO2 and NO2 in the region under study. Our work shows that much of the signal identifying the sources of NO2 and SO2 emissions resides in shorter timescales (hourly to daily) due to short-term variation of concentrations and that longer-term averages in data collection may lose the information needed to identify local sources. However, the methodology identifies stations mainly influenced by seasonality, if larger timescales (weekly to monthly) are considered. We have performed the first dissimilarity analysis based on gridded air quality model output and have shown that the methodology is capable of generating maps of subregions within which a single station will represent the entire subregion, to a given level of dissimilarity. We have also shown that our approach is capable of identifying different sampling methodologies as well as outliers (stations' time series which are markedly different from all others in a given dataset).

Stable sulfur isotope measurements to trace the fate of SO2 in the Athabasca oil sands region

“…A range of 17 to 36 % of global sulfate production can be attributed to this pathway (Chin et al, 2000;Sofen et al, 2011;Berglen, 2004). Heterogeneous oxidation of SO 2 primarily occurs in cloud droplets, although oxidation on the surface of aerosols can be important regionally (Chin and Jacob, 1996).…”

Clustering analysis of very large measurement and model datasets on high performance computing platforms

“…Using the previous serial version of the algorithm with the hardware available at the time, it was not possible to cluster more than approximately 100 × 100 or 10,000 or approximately 3% of the model domain within the wallclock time limits imposed on that system (4.5 hours; Soares et al 2018). The new software improves upon the old architecture in 2 ways: replacing the dissimilarity matrix with a list of pqueues, and parallelization with the MPI and OpenMPI APIs.…”

“…However, the large-scale analysis of air-quality model output to determine regions of similar chemistry is hampered by the poor scalability of existing analysis algorithms (Soares et al, 2018;Saxena et al, 2017).…”

Optimal Design of a Surface Precipitation Network in Canada

“…For each of these OND goals, specialized techniques have been developed. For example, Soares et al (2018) applied hierarchical clustering methods to identify redundant air quality monitoring stations or to design a network using numerical model outputs for the oil sands region in northern Alberta while Kalinić et al (2021) selected potential wind buoy locations over the Adriatic Sea by K-means clustering of model reanalysis data. Observation system experiment (OSE) or data denial experiments compare analysis or forecasts with and without a subset of observations assimilated.…”