Optimization of Deep Learning Precipitation Models Using Categorical Binary Metrics

Abstract

This work introduces a methodology for optimizing neural network models using a combination of continuous and categorical binary indices in the context of precipitation forecasting. Probability of detection and false alarm rate are popular metrics used in the verification of precipitation models. However, machine learning models trained using gradient descent cannot be optimized based on these metrics, as they are not differentiable. We propose an alternative formulation for these categorical indices that are differentiable and we demonstrate how they can be used to optimize the skill of precipitation neural network models defined as a multiobjective optimization problem. To our knowledge, this is the first proposal of a methodology for optimizing weather neural network models based on categorical indices. Plain Language Summary Deep neural networks have recently demonstrated great versatilityand an unprecedented capacity to model complex problems. In weather modeling, these algorithms have been applied to solve different problems. This is a promising area of research, given the availability of large volumes of weather data and increasingly powerful computers. Neural network models can learn to solve problems based on a metric, which the model tries to optimize. However, the quality of weather models is measured using a large variety of metrics, which can be a challenge when choosing which metric the model should optimize. In the case of precipitation, categorical binary metrics are a popular choice to assess the quality of a model. These metrics reduce precipitation to a "yes" or "no" event, and the results of the predicting model can be compared with the actual observations. This method is simple, yet powerful and a large number of indices and statistics have been developed to assess different aspects of the quality of precipitation models. As precipitation models are commonly assessed using these categorical binary metrics, it would be very convenient to optimize models based on them. Unfortunately, the mathematical nature of these metrics makes them unsuitable for optimizing deep learning models. In this work we present an alternative formulation for these categorical binary indices which can be used to train models. We demonstrate how a deep learning model can be trained to generate better quality precipitation data.

The AI gambit: leveraging artificial intelligence to combat climate change—opportunities, challenges, and recommendations

“…The ability of AI to process enormous amounts of non-structured, multi-dimensional data using sophisticated optimisation techniques is already facilitating the understanding of high-dimensional climate datasets and forecasting of future trends (Huntingford et al 2019 ). AI techniques have been used to forecast global mean temperature changes (Ise and Oba 2019 ; Cifuentes et al 2020 ); predict climactic and oceanic phenomena such as El Niño (Ham et al 2019 ), cloud systems (Rasp et al 2018 ), and tropical instability waves (Zheng et al 2020 ); better understand aspects of the weather system—like rainfall, generally (Sønderby et al 2020 ; Larraondo et al 2020 ) and in specific locales, such as Malaysia (Ridwan et al 2020 )—and their knock-on consequences, like water demand (Shrestha et al 2020 ; Xenochristou et al 2020 ). AI tools can also help anticipate the extreme weather events that are more common as a result of global climate change, for example heavy rain damage (Choi et al 2018 ) and wildfires (Jaafari et al 2019 ), and other downstream consequences, such as patterns of human migration (Robinson and Dilkina 2018 ).…”

The AI Gambit — Leveraging Artificial Intelligence to Combat Climate Change: Opportunities, Challenges, and Recommendations

“…The ability of AI to process enormous amounts of non-structured, multi-dimensional data using sophisticated optimisation techniques is already facilitating the understanding of high-dimensional climate datasets and forecasting of future trends (Huntingford et al 2019). AI techniques have been used to forecast global mean temperature changes (Ise and Oba 2019;Cifuentes et al 2020); predict climactic and oceanic phenomena such as El Niño (Ham, Kim, and Luo 2019), cloud systems (Rasp, Pritchard, and Gentine 2018), and tropical instability waves (Zheng et al 2020); better understand aspects of the weather system -like rainfall, generally (Sønderby et al 2020;Larraondo et al 2020) and in specific locales, such as Malaysia (Ridwan et al 2020) -and their knock-on consequences, like water demand (Shrestha, Manandhar, and Shrestha 2020;Xenochristou et al 2020).…”

Deep learning‐based precipitation bias correction approach for Yin–He global spectral model

“…for which means element-wise multiplication (P. R. Larraondo et al, 2020). For logical operations like observed < α ð Þor observed > α ð Þ , it is 1 if the statement is true, otherwise it is 0.…”

“…These metrics are noncontinuous, non-differentiable, and unsuitable for optimizing deep learning models. P. R. Larraondo et al (2020) proposed an alternative formulation for these categorical binary indices, which are defined as follows:…”

“…Xu et al (2019) proposed a novel deep convolution framework entitled ordinal boosting autoencoder (OBA) that is appropriate for the bias correction during the numerical precipitation prediction process. Accordingly, P. R. Larraondo et al (2020) introduced a methodology for optimizing neural network models using a combination of continuous and categorical binary indices in the context of precipitation forecasting. Moreover, Dupuy et al (2020) suggested a U-Net architecture to perform cloud cover forecast post-processing, while Trebing et al (2021) proposed a U-Net-based model to nowcast Netherlands' precipitation.…”