AQ-Bench: a benchmark dataset for machine learning on global air quality metrics

Abstract

Abstract. With the AQ-Bench dataset, we contribute to the recent developments towards shared data usage and machine learning methods in the field of environmental science. The dataset presented here enables researchers to relate global air quality metrics to easy-access metadata and to explore different machine learning methods for obtaining estimates of air quality based on this metadata. AQ-Bench contains a unique collection of aggregated air quality data from the years 2010–2014 and metadata at more than 5500 air quality monitoring stations all over the world, provided by the first Tropospheric Ozone Assessment Report (TOAR). It focuses in particular on metrics of tropospheric ozone, which has a detrimental effect on climate, human morbidity and mortality, as well as crop yields. The purpose of this dataset is to produce estimates of various long-term ozone metrics based on time-independent local site conditions. We combine this task with a suitable evaluation metric. Baseline scores obtained from a linear regression method, a fully connected neural network and random forest are provided for reference and validation. AQ-Bench offers a low-threshold entrance for all machine learners with an interest in environmental science and for atmospheric scientists who are interested in applying machine learning techniques. It enables them to start with a real-world problem relevant to humans and nature. The dataset and introductory machine learning code are available at https://doi.org/10.23728/b2share.30d42b5a87344e82855a486bf2123e9f (Betancourt et al., 2020) and https://gitlab.version.fz-juelich.de/esde/machine-learning/aq-bench (Betancourt et al., 2021). AQ-Bench thus provides a blueprint for environmental benchmark datasets as well as an example for data re-use according to the FAIR principles.

Global, high-resolution mapping of tropospheric ozone – explainable machine learning and impact of uncertainties

“…A 100 random forest (Breiman, 2001) models are the state of the art for structured data (Lundberg et al, 2020). Random forest was also shown to outperform linear regression and a shallow neural network in predicting average ozone on the AQ-Bench dataset (Betancourt et al, 2021b).…”

“…We rely on the independent data split of AQ-Bench as provided by Betancourt et al (2021b). Here, stations with a distance of more than 50 km are considered independent of each other.…”

“…To prove a machine learning models' robustness, cross validation can be applied. We reserve 20 % of the AQ-Bench dataset for testing the final model, relying on the independent split of Betancourt et al (2021b). We split the remaining 80 % into four independent cross validation folds of 20 % each.…”

“…We split the remaining 80 % into four independent cross validation folds of 20 % each. Like Betancourt et al (2021b), we assume that air quality measurement stations with a distance of at least 50 km are independent of each other. We, therefore, produce the cross validation folds with a two-step approach.…”

“…Ozone and ozone precursor datasets which can be used as training data for machine learning models are being increasingly made available as FAIR (Wilkinson et al, 2016) and open data. One of these datasets is AQ-Bench ('air quality benchmark dataset, ' Betancourt et al, 2021b), a dataset for machine learning on global ozone metrics which also serves as training data for this mapping study.…”