Pace v0.1: A Python-based Performance-Portable Implementation of the FV3 Dynamical Core

Abstract

Abstract. Progress in leveraging current and emerging high-performance computing infrastructures using traditional weather and climate models has been slow. This has become known more broadly as the software productivity gap. With the end of Moore's Law driving forward rapid specialization of hardware architectures, building simulation codes on a low-level language with hardware specific optimizations is a significant risk. As a solution, we present Pace, an implementation of the nonhydrostatic FV3 dynamical core which is entirely Python-based. In order to achieve high performance on a diverse set of hardware architectures, Pace is written using the GT4Py domain-specific language. We demonstrate that with this approach we can achieve portability and performance, while significantly improving the readability and maintainability of the code as compared to the Fortran reference implementation. We show that Pace can run at scale on leadership-class supercomputers and achieve performance speeds 3.5–4 times faster than the Fortran code on GPU-accelerated supercomputers. Furthermore, we demonstrate how a Python-based simulation code facilitates existing or enables entirely new use-cases and workflows. Pace demonstrates how a high-level language can insulate us from disruptive changes, provide a more productive development environment, and facilitate the integration with new technologies such as machine learning.

Implementation of the Novel Duo‐Grid in GFDL's FV3 Dynamical Core

“…One intrinsic computational advantage of the Duo‐Grid is that it eliminates the need for edge handling within the solver algorithms, especially the advection scheme, and thereby can maintain steady computational work throughout the domain during the integration. This can yield significant performance gains on Graphical Programming Unit (GPU) processors, since the edge handling is currently a significant bottleneck on these systems (Dahm et al., 2022). Future work will also describe full‐physics simulations with the Duo‐Grid.…”

Neural Network Parameterization of Subgrid‐Scale Physics From a Realistic Geography Global Storm‐Resolving Simulation

“…Framing parameterizations in terms of machine learning algorithms allows the use of the rapidly growing suite of software tools for training and inference, potentially leading to improvements in computational efficiency at runtime on modern computing architectures like GPUs. This would be particularly advantageous if machine learning parameterizations could run in concert with a dynamical core that runs efficiently on GPUs (e.g., Dahm et al, 2022). It can also enable novel strategies for the prediction of subgrid-scale phenomena, such as the use of data from neighboring columns (P. .…”

Domain-Specific Implementation of High-Order Discontinuous Galerkin Methods in Spherical Geometry

“…G4GT, now discontinued, was a first prototype to extend these tools for finite element problems. GT4Py is a Python layer above GridTools and thus targets the same FD/FV problems, for example [10]. GT4Py/GridTools now have been used for the porting of production models, among them FVM [38], FV3 [7] and ICON [5].…”