Causally‐Informed Deep Learning to Improve Climate Models and Projections

Iglesias‐Suarez, Fernando; Gentine, Pierre; Solino‐Fernandez, Breixo; Beucler, Tom; Pritchard, Michael; Runge, Jakob; Eyring, Veronika

doi:10.1029/2023jd039202

Cited by 5 publications

(1 citation statement)

References 108 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Respecting the causal direction of time has shown to be effective in training PINNs for chaotic systems where previous approaches failed [51]. Furthermore, coupling causal discovery to identify the causal drivers in climate models before applying DL algorithms improved performance and interpretability [52,53]. Causally constrained recurrent NNs more accurately reflect underlying processes and were shown to enhance our understanding of methane in wetlands [54].…”

Section: Introductionmentioning

confidence: 99%

Causal hybrid modeling with double machine learning—applications in carbon flux modeling

Cohrs,

Varando,

Carvalhais

et al. 2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Hybrid modeling integrates machine learning with scientific knowledge to enhance interpretability, generalization, and adherence to natural laws. Nevertheless, equifinality and regularization biases pose challenges in hybrid modeling to achieve these purposes. This paper introduces a novel approach to estimating hybrid models via a causal inference framework, specifically employing Double Machine Learning (DML) to estimate causal effects. We showcase its use for the Earth sciences on two problems related to carbon dioxide fluxes. In the Q10 model, we demonstrate that DML-based hybrid modeling is superior in estimating causal parameters over end-to-end deep neural network (DNN) approaches, proving efficiency, robustness to bias from regularization methods, and circumventing equifinality. Our approach, applied to carbon flux partitioning, exhibits flexibility in accommodating heterogeneous causal effects. The study emphasizes the necessity of explicitly defining causal graphs and relationships, advocating for this as a general best practice. We encourage the continued exploration of causality in hybrid models for more interpretable and trustworthy results in knowledge-guided machine learning.

show abstract

Section: Introductionmentioning

confidence: 99%

Causal hybrid modeling with double machine learning—applications in carbon flux modeling

Cohrs,

Varando,

Carvalhais

et al. 2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

show abstract

Data Imbalance, Uncertainty Quantification, and Transfer Learning in Data‐Driven Parameterizations: Lessons From the Emulation of Gravity Wave Momentum Transport in WACCM

Sun,

Pahlavan,

Chattopadhyay

et al. 2024

J Adv Model Earth Syst

View full text Add to dashboard Cite

Neural networks (NNs) are increasingly used for data‐driven subgrid‐scale parameterizations in weather and climate models. While NNs are powerful tools for learning complex non‐linear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are (a) data imbalance related to learning rare, often large‐amplitude, samples; (b) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and (c) generalization to other climates, for example, those with different radiative forcings. Here, we examine the performance of methods for addressing these challenges using NN‐based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics‐based gravity wave (GW) parameterizations as a test case. WACCM has complex, state‐of‐the‐art parameterizations for orography‐, convection‐, and front‐driven GWs. Convection‐ and orography‐driven GWs have significant data imbalance due to the absence of convection or orography in most grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto‐encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria for identifying when an NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4 × CO2). However, their performance is significantly improved by applying transfer learning, for example, re‐training only one layer using ∼1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data‐driven parameterizations for various processes, including (but not limited to) GWs.

show abstract

Interpretable Multiscale Machine Learning‐Based Parameterizations of Convection for ICON

Heuer,

Schwabe,

Gentine

et al. 2024

J Adv Model Earth Syst

View full text Add to dashboard Cite

Machine learning (ML)‐based parameterizations have been developed for Earth System Models (ESMs) with the goal to better represent subgrid‐scale processes or to accelerate computations. ML‐based parameterizations within hybrid ESMs have successfully learned subgrid‐scale processes from short high‐resolution simulations. However, most studies used a particular ML method to parameterize the subgrid tendencies or fluxes originating from the compound effect of various small‐scale processes (e.g., radiation, convection, gravity waves) in mostly idealized settings or from superparameterizations. Here, we use a filtering technique to explicitly separate convection from these processes in simulations with the Icosahedral Non‐hydrostatic modeling framework (ICON) in a realistic setting and benchmark various ML algorithms against each other offline. We discover that an unablated U‐Net, while showing the best offline performance, learns reverse causal relations between convective precipitation and subgrid fluxes. While we were able to connect the learned relations of the U‐Net to physical processes this was not possible for the non‐deep learning‐based Gradient Boosted Trees. The ML algorithms are then coupled online to the host ICON model. Our best online performing model, an ablated U‐Net excluding precipitating tracer species, indicates higher agreement for simulated precipitation extremes and mean with the high‐resolution simulation compared to the traditional scheme. However, a smoothing bias is introduced both in water vapor path and mean precipitation. Online, the ablated U‐Net significantly improves stability compared to the non‐ablated U‐Net and runs stable for the full simulation period of 180 days. Our results hint to the potential to significantly reduce systematic errors with hybrid ESMs.

show abstract

Causally‐Informed Deep Learning to Improve Climate Models and Projections

Cited by 5 publications

References 108 publications

Causal hybrid modeling with double machine learning—applications in carbon flux modeling

Causal hybrid modeling with double machine learning—applications in carbon flux modeling

Data Imbalance, Uncertainty Quantification, and Transfer Learning in Data‐Driven Parameterizations: Lessons From the Emulation of Gravity Wave Momentum Transport in WACCM

Interpretable Multiscale Machine Learning‐Based Parameterizations of Convection for ICON

Contact Info

Product

Resources

About