Data‐Driven Super‐Parameterization Using Deep Learning: Experimentation With Multiscale Lorenz 96 Systems and Transfer Learning

Abstract

To make weather and climate models computationally affordable, small-scale processes are usually represented in terms of the large-scale, explicitly resolved processes using physics-based/ semi-empirical parameterization schemes. Another approach, computationally more demanding but often more accurate, is super-parameterization (SP). SP involves integrating the equations of small-scale processes on high-resolution grids embedded within the low-resolution grid of large-scale processes. Recently, studies have used machine learning (ML) to develop data-driven parameterization (DD-P) schemes. Here, we propose a new approach, data-driven SP (DD-SP), in which the equations of the small-scale processes are integrated data-drivenly (thus inexpensively) using ML methods such as recurrent neural networks. Employing multiscale Lorenz 96 systems as the testbed, we compare the cost and accuracy (in terms of both short-term prediction and long-term statistics) of parameterized low-resolution (PLR) SP, DD-P, and DD-SP models. We show that with the same computational cost, DD-SP substantially outperforms PLR and is more accurate than DD-P, particularly when scale separation is lacking. DD-SP is much cheaper than SP, yet its accuracy is the same in reproducing long-term statistics (climate prediction) and often comparable in short-term forecasting (weather prediction). We also investigate generalization: when models trained on data from one system are applied to a more chaotic system, we find that models often do not generalize, particularly when short-term prediction accuracies are examined. However, we show that transfer learning, which involves retraining the data-driven model with a small amount of data from the new system, significantly improves generalization. Potential applications of DD-SP and transfer learning in climate/weather modeling are discussed. Plain Language Summary The weather/climate system involves intertwined physical processes acting on scales from centimeters (or even smaller) to tens of thousands of kilometers. Most weather/climate models used in practice include parameterization schemes that relate small-scale processes, which are not explicitly resolved (due to coarse spatiotemporal resolution), to large-scale processes that are resolved. Recently, studies have explored using machine learning for data-driven parameterization (DD-P) of small-scale (subgrid) processes. Here, we first introduce a novel way to leverage recent advances in deep learning to improve the modeling of subgrid processes. In this approach, called data-driven super-parameterization (DD-SP), deep learning is used for fast, data-driven integration of equations of small-scale processes, while other equations are integrated using conventional numerical methods. Employing a relatively simple chaotic system, we show the advantages of DD-SP over DD-P and conventional parameterizations. Second, we examine how these data-driven models generalize (extrapolate) from one system to other (e.g., more chaotic) systems. We demonstrate that these mod...

Deep Learning for Geophysics: Current and Future Trends

“…The first attempts started with simple FCNN methods followed by complex networks, such as CNN, RNN, and GAN models. With respect to the training set, early works used end-to-end training borrowed from the computer vision area, which requires a large number of annotated labels, while recent works have started to consider unsupervised learning (He et al, 2018) and the combination of DL with a physical model (Chattopadhyay et al, 2020;Wu & McMechan, 2019). In 2020, more works focused on the uncertainty of DL methods (Cao et al, 2020;Grana et al, 2020;Mousavi & Beroza, 2020a).…”

“…For instance, on the STanford EArthquake Data set (STEAD), the earthquake detection accuracy is improved to 100% compared to 91% accuracy of the traditional STA/LTA (short time average over long time average) method (Mousavi, Zhu, Sheng, et al, 2019, Mousavi et al, 2020. DL makes characterizing the earth with high resolution on a large scale possible (Chattopadhyay et al, 2020;Chen et al, 2019;Zhang, Stanev, & Grayek, 2020). DL can even be used for discovering physical concepts (Iten et al, 2020), such as the solar system is heliocentric.…”

Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels

“…Further opportunities and perspectives for the use of data-driven methods for geosciences may be found in the reviews of [32,36]. Before proceeding, we note that a vast majority of the data-driven developments for geophysical forecasting have involved the use of variants of deep learning methods, for example ResNets [37], CapsuleNets [22], U-Nets [31], long short-term memory networks (LSTMs) [33], convolutional-LSTMs [34], neural ordinary differential equations (ODEs) [38,39] and local or global fully connected deep neural networks [32]. These methods, while exceptionally powerful in learning complex functions, hamper interpretability and require large computational resources for optimization.…”

Modeling Noise: Paths toward AI-Enabled Stochastic Earth System Models and Parameterizations

“…A third approach is a Linear Inverse Modeling framework (Newman et al, 2003;Martinez-Villalobos et al, 2017), where the predictive modes are represented as covariance functions in a reduced space (e.g., functions of PCAs). We can also model the system with reduced complexity and represent higher complexity processes as AI-driven stochastic processes (Chattopadhyay et al, 2020;Crommelin and Edeling, 2020;Alcala and Timofeyev, 2020;Leinonen et al, 2020). To characterize noise relevant for predicting high-frequency signals, convection-resolving simulations such as the DYAMOND ensemble (Stevens et al, 2019) provide comprehensive data coverage to characterize variability in small-scale processes (Christensen, 2020).…”