CPMIP: measurements of real computational performance of Earth system models in CMIP6

Abstract

Abstract. A climate model represents a multitude of processes on a variety of timescales and space scales: a canonical example of multi-physics multi-scale modeling. The underlying climate system is physically characterized by sensitive dependence on initial conditions, and natural stochastic variability, so very long integrations are needed to extract signals of climate change. Algorithms generally possess weak scaling and can be I/O and/or memory-bound. Such weakscaling, I/O, and memory-bound multi-physics codes present particular challenges to computational performance.Traditional metrics of computational efficiency such as performance counters and scaling curves do not tell us enough about real sustained performance from climate models on different machines. They also do not provide a satisfactory basis for comparative information across models.We introduce a set of metrics that can be used for the study of computational performance of climate (and Earth system) models. These measures do not require specialized software or specific hardware counters, and should be accessible to anyone. They are independent of platform and underlying parallel programming models. We show how these metrics can be used to measure actually attained performance of Earth system models on different machines, and identify the most fruitful areas of research and development for performance engineering.We present results for these measures for a diverse suite of models from several modeling centers, and propose to use these measures as a basis for a CPMIP, a computational performance model intercomparison project (MIP).

The GFDL Global Atmosphere and Land Model AM4.0/LM4.0: 2. Model Description, Sensitivity Studies, and Tuning Strategies

“…With perfect scaling, CHSY will not change as one increases parallelism. However, as scaling efficiency drops, CHSY will increase and it may not be the most economical use of one's resources to run at the maximum SYPD possible (see Balaji et al, 2017, for a full analysis of these metrics).…”

Requirements for a global data infrastructure in support of CMIP6

“…CMIP6 (Eyring et al, 2016a), the latest Coupled Model Intercomparison Project (CMIP), can trace its genealogy back to the "Charney report" (Charney et al, 1979). This seminal report on the links between CO 2 and climate was an authoritative summary of the state of the science at the time and produced findings that have stood the test of time (Bony et al, 2013). It is often noted (e.g Andrews et al, 2012) that the range and uncertainty bounds on equilibrium climate sensitivity generated in this report have not fundamentally changed, despite the enormous increase in resources devoted to analysing the problem in decades since (e.g Knutti et al, 2017) Beyond its enduring findings on climate sensitivity, the Charney report also gave rise to a methodology for the treatment of uncertainties and gaps in understanding, which has been equally influential, and is in fact the basis of CMIP itself.…”

LIVVkit: An extensible, python‐based, land ice verification and validation toolkit for ice sheet models

“…For CISM 2.0+ regression data sets, performance tests include the time‐evolving ice sheet dome test executed for a range of problem sizes, and each size is repeated using a different number of processors. Each test is repeated eleven times in order to capture the performance variability, which is a problem that has been encountered in other Earth system modeling components at all job sizes [e.g., Balaji et al ., ].…”