From proof‐of‐concept to proof‐of‐value: Approaching third‐party data to operational workflows of national meteorological services

Abstract

National meteorological services (NMS) are limited by practical and financial boundaries in the number of official meteorological measurements that it can collect. This means that large regions are often unobserved. These gaps can be filled by novel data sources, including measurements from personal weather stations that are owned and operated by amateur citizen scientists, or opportunistic sensing from devices that are not designed to measure meteorological variables, like commercial microwave links (radio connections between mobile phone towers). These types of data are known as “third‐party data” (3PD) as they are not owned or operated by NMS or research institutes (e.g., university, government department). Demonstration of the quality and value of these novel data sources is an active area of research. NMS, like the Royal Netherlands Meteorological Institute (KNMI), are faced with some unique challenges when it comes to transferring research to operations. KNMI is in the early stages of developing an operational pipeline for 3PD. We outline some use cases where we have demonstrated the quality of 3PD. We discuss our experiences with some of these challenges that can occur when transferring between proof‐of‐concepts on 3PD developed in research settings into real operational workflows providing valuable services. Hence, in this work we introduce our third‐party data life cycle, in which we provide an integral overview of this transitioning process considering business and social aspects, technical feasibility assessments, the importance of quality control, and aspects related to data integration and alignment with the existing official data sources. We also reflect on how these potential new applications could fit into KNMIs long‐term strategies and contribute to the high‐resolution weather forecast and early warning issuing. We hope that sharing these experiences will provide some general guidelines to organizations in need of providing new services stemming from 3PD and transform them into “daily business.”

EURADCLIM: the European climatological high-resolution gauge-adjusted radar precipitation dataset

“…-Specifically, third-party data could be employed after appropriate quality-control (and retrieval) algorithms have been applied. This also requires efforts from NMHSs to bring these from research to operations (Garcia-Marti et al, 2023). Examples of such third-party data sources are rain gauge data from personal weather stations (PWSs) connected to the Internet (De Vos et al, 2019;Graf et al, 2021) and commercial microwave link (CML) data (Messer et al, 2006;Leijnse et al, 2007;Overeem et al, 2016;Graf et al, 2020Graf et al, , 2021.…”

Pushing the Nyquist-Shannon limit: exploiting multi-fidelity observations to provide high-resolution weather maps for climate services, early warning centres and weather forecasts

“…In recent years, there has been an intense research activity around the usage and incorporation of crowdsourced data in the climate sciences. Substantial efforts have been carried out to perform quality assessment for crowdsourced data [16], fitting these novel observations into the numerical weather predictions [21,46], or defining workflows to transform these observations into valuable new products and services for NMS [18].…”

“…These observations are sent to the KNMI for its subsequent inclusion in the fundamental weather products and services. The spatial distribution of the network is such so that it balances financial considerations with a good coverage for large-scale phenomena, but it implies that large local regions remain unobserved [18].…”

EURADCLIM: The European climatological high-resolution gauge-adjusted radar precipitation dataset

“…-Specifically, third party data could be employed after appropriate quality control (and retrieval) algorithms have been applied. This also requires efforts from NMHS to bring these from research to operations (Garcia-Marti et al, 2022).…”