Unprecedented cloud resolution in a GPU-enabled full-physics atmospheric climate simulation on OLCF’s summit supercomputer

Abstract

Clouds represent a key uncertainty in future climate projection. While explicit cloud resolution remains beyond our computational grasp for global climate, we can incorporate important cloud effects through a computational middle ground called the Multi-scale Modeling Framework (MMF), also known as Super Parameterization. This algorithmic approach embeds high-resolution Cloud Resolving Models (CRMs) to represent moist convective processes within each grid column in a Global Climate Model (GCM). The MMF code requires no parallel data transfers and provides a self-contained target for acceleration. This study investigates the performance of the Energy Exascale Earth System Model-MMF (E3SM-MMF) code on the OLCF Summit supercomputer at an unprecedented scale of simulation. Hundreds of kernels in the roughly 10K lines of code in the E3SM-MMF CRM were ported to GPUs with OpenACC directives. A high-resolution benchmark using 4600 nodes on Summit demonstrates the computational capability of the GPU-enabled E3SM-MMF code in a full physics climate simulation.

Improving Stratocumulus Cloud Amounts in a 200‐m Resolution Multi‐Scale Modeling Framework Through Tuning of Its Interior Physics

“…Each tuning experiment was run for 6 months, from January to June, using seasonally varying climatological conditions based on the years 2005–2014. This ambitious ensemble was made possible by ongoing development to enhance the throughput of E3SM‐MMF, which includes code refactoring to leverage GPU hardware acceleration on the Oak Ridge Leadership Computing Facility (OLCF) Summit machine (Norman et al., 2019).…”

Comment on gmd-2022-141

“…The advent of heterogeneous super-computing platforms, together with the increasing computing power and low energy to performance ratio of GPUs, has motivated the use of GPUs to accelerate climate and weather models. In recent years, numerous studies have reported successful GPU porting of full or partial weather and climate models with improved performances (Hanappe et al, 2011;Xu et al, 2015;Yuan et al, 2020;Zhang et al, 2020;Bieringer et al, 2021;Mielikainen et al, 2011;Michalakes and Vachharajani, 2008;Shimokawabe et al, 2010;Govett et al, 2017;Li and Van Roekel, 2021;Xiao et al, 2013;, Norman et al, 2022Bertagna et al, 2020).…”

Evaluating Mesoscale Convective Systems Over the US in Conventional and Multiscale Modeling Framework Configurations of E3SMv1

“…High-resolution and MMF methods are both computationally expensive, and the cost of using them simultaneously for long-term climate simulations with Earth system models (e.g., Coupled Model Intercomparison Project experiments) is prohibitive. Nevertheless, it has recently been shown that a GPU accelerated MMF model can be competitive with traditionally parameterized models in terms of run-time and CPU-hours (Norman et al, 2022). To understand which approach independently provides a better simulation of overall precipitation over CONUS, Kooperman et al (2022) compared high-resolution and MMF experiments that were configured to use roughly equivalent computational cost in E3SMv1.…”