Deep-learning-based precipitation observation quality control

Sha, Yingkai; Gagne, David John; West, Gregory; Stull, Roland B.

doi:10.1175/jtech-d-20-0081.1

Cited by 6 publications

(3 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Simonyan et al 15 initially proposed the saliency map method as a visualization technique to explain the neural network function mapping, specifically, the extent to which inputs contribute to network output. Due to their effectiveness, explainable deep learning methods have been widely applied to the geosciences and especially to understand climate science and translate to impacts, for example, in spatial drought prediction 16 , satellite-based PM2.5 (air pollution) measurements 17 , crop yields 18 , species distribution models 19 , analysis of hailstorms 20 , hydro-climatological process modeling 21 , precipitation quality control 22 and climate drivers for global temperature 23 , and to localize pest insects in agricultural application 24 . Ham et al 13 used a saliency map to analyze which regions contributed most in predicting the Niño 3.4 index using their neural network.…”

Section: Introductionmentioning

confidence: 99%

Explainable deep learning for insights in El Niño and river flows

Liu

Duffy

et al. 2023

Nat Commun

View full text Add to dashboard Cite

The El Niño Southern Oscillation (ENSO) is a semi-periodic fluctuation in sea surface temperature (SST) over the tropical central and eastern Pacific Ocean that influences interannual variability in regional hydrology across the world through long-range dependence or teleconnections. Recent research has demonstrated the value of Deep Learning (DL) methods for improving ENSO prediction as well as Complex Networks (CN) for understanding teleconnections. However, gaps in predictive understanding of ENSO-driven river flows include the black box nature of DL, the use of simple ENSO indices to describe a complex phenomenon and translating DL-based ENSO predictions to river flow predictions. Here we show that eXplainable DL (XDL) methods, based on saliency maps, can extract interpretable predictive information contained in global SST and discover SST information regions and dependence structures relevant for river flows which, in tandem with climate network constructions, enable improved predictive understanding. Our results reveal additional information content in global SST beyond ENSO indices, develop understanding of how SSTs influence river flows, and generate improved river flow prediction, including uncertainty estimation. Observations, reanalysis data, and earth system model simulations are used to demonstrate the value of the XDL-CN based methods for future interannual and decadal scale climate projections.

show abstract

Section: Introductionmentioning

confidence: 99%

Explainable deep learning for insights in El Niño and river flows

Liu

Duffy

et al. 2023

Nat Commun

View full text Add to dashboard Cite

show abstract

“…Would deeplearning-based quality control of observations be included within a hybrid system (e.g. Sha et al, 2021)? Other potential inclusions into the class of hybrid forecasting systems are machine-learning aided data assimilation (Boucher et al, 2019;He et al, 2020;Liu et al, 2021), and commonly postprocessing (e.g.…”

Section: Introductionmentioning

confidence: 99%

Using a long short-term memory (LSTM) neural network to boost river streamflow forecasts over the western United States

Hunt

Matthews

Pappenberger

et al. 2022

Hydrol. Earth Syst. Sci.

View full text Add to dashboard Cite

Abstract. Accurate river streamflow forecasts are a vital tool in the fields of water security, flood preparation and agriculture, as well as in industry more generally. Traditional physics-based models used to produce streamflow forecasts have become increasingly sophisticated, with forecasts improving accordingly. However, the development of such models is often bound by two soft limits: empiricism – many physical relationships are represented empirical formulae; and data sparsity – long time series of observational data are often required for the calibration of these models. Artificial neural networks have previously been shown to be highly effective at simulating non-linear systems where knowledge of the underlying physical relationships is incomplete. However, they also suffer from issues related to data sparsity. Recently, hybrid forecasting systems, which combine the traditional physics-based approach with statistical forecasting techniques, have been investigated for use in hydrological applications. In this study, we test the efficacy of a type of neural network, the long short-term memory (LSTM), at predicting streamflow at 10 river gauge stations across various climatic regions of the western United States. The LSTM is trained on the catchment-mean meteorological and hydrological variables from the ERA5 and Global Flood Awareness System (GloFAS)–ERA5 reanalyses as well as historical streamflow observations. The performance of these hybrid forecasts is evaluated and compared with the performance of both raw and bias-corrected output from the Copernicus Emergency Management Service (CEMS) physics-based GloFAS. Two periods are considered, a testing phase (June 2019 to June 2020), during which the models were fed with ERA5 data to investigate how well they simulated streamflow at the 10 stations, and an operational phase (September 2020 to October 2021), during which the models were fed forecast variables from the European Centre for Medium-Range Weather Forecasts (ECMWF) Integrated Forecasting System (IFS), to investigate how well they could predict streamflow at lead times of up to 10 d. Implications and potential improvements to this work are discussed. In summary, this is the first time an LSTM has been used in a hybrid system to create a medium-range streamflow forecast, and in beating established physics-based models, shows promise for the future of neural networks in hydrological forecasting.

show abstract

“…The ARF method incorporates the optimization ability of the artificial fish swarm algorithm and the random forest regression function to provide quality control for multiple surface air temperature stations. Sha presented a novel approach for the automated quality control of precipitation for a sparse station observation network within the complex terrain of British Columbia, Canada (Sha et al ., 2021). The QC approach used convolutional neural networks (CNNs) to classify bad observation values, incorporating a multi classifier ensemble to achieve better quality control performance.…”

Section: Introductionmentioning

confidence: 99%

Research on quality control methods for surface temperature observations via spatial correlation analysis

Xiong

Jiang

Tang

et al. 2022

Intl Journal of Climatology

View full text Add to dashboard Cite

This article proposed a new quality control method (CS-MSF) to identify potential outliers in the surface temperature observations. The CS-MSF method employed cosine similarity and moving surface fitting to obtain the estimated value of the target station. For the regions with complex terrain and low weather station density, another quality control method (CE-GBDT) was employed to compensate for the shortcomings of CS-MSF. Compared to the spatial regression test method (SRT) and inverse distance weighting method (IDW), the results indicated that CS-MSF outperformed SRT and IDW in all the cases. And CE-GBDT was superior to the other methods for the regions with complex terrain and low weather station density. The comparison results led to the recommendation that the two proposed methods are effective quality control methods in identifying the seeded errors for the surface temperature observations.

show abstract

Deep-learning-based precipitation observation quality control

Cited by 6 publications

References 55 publications

Explainable deep learning for insights in El Niño and river flows

Explainable deep learning for insights in El Niño and river flows

Using a long short-term memory (LSTM) neural network to boost river streamflow forecasts over the western United States

Research on quality control methods for surface temperature observations via spatial correlation analysis

Contact Info

Product

Resources

About