Towards improved and more routine Earth system model evaluation in CMIP

Abstract

Abstract. The Coupled Model Intercomparison Project (CMIP) has successfully provided the climate community with a rich collection of simulation output from Earth system models (ESMs) that can be used to understand past climate changes and make projections and uncertainty estimates of the future. Confidence in ESMs can be gained because the models are based on physical principles and reproduce many important aspects of observed climate. More research is required to identify the processes that are most responsible for systematic biases and the magnitude and uncertainty of future projections so that more relevant performance tests can be developed. At the same time, there are many aspects of ESM evaluation that are well established and considered an essential part of systematic evaluation but have been implemented ad hoc with little community coordination. Given the diversity and complexity of ESM analysis, we argue that the CMIP community has reached a critical juncture at which many baseline aspects of model evaluation need to be performed much more efficiently and consistently. Here, we provide a perspective and viewpoint on how a more systematic, open, and rapid performance assessment of the large and diverse number of models that will participate in current and future phases of CMIP can be achieved, and announce our intention to implement such a system for CMIP6. Accomplishing this could also free up valuable resources as many scientists are frequently "re-inventing the wheel" by re-writing analysis routines for well-established analysis methods. A more systematic approach for the community would be to develop and apply evaluation tools that are based on the latest scientific knowledge and observational reference, are well suited for routine use, and provide a wide range of diagnostics and performance metrics that comprehensively characterize model behaviour as soon as the output is published to the Earth System Grid Federation (ESGF). The CMIP infrastructure enforces data standards and conventions for model output and documentation accessible via the ESGF, additionally publishing observations (obs4MIPs) and reanalyses (ana4MIPs) for modelPublished by Copernicus Publications on behalf of the European Geosciences Union. V. Eyring et al.: Towards improved and more routine Earth system model evaluation in CMIPintercomparison projects using the same data structure and organization as the ESM output. This largely facilitates routine evaluation of the ESMs, but to be able to process the data automatically alongside the ESGF, the infrastructure needs to be extended with processing capabilities at the ESGF data nodes where the evaluation tools can be executed on a routine basis. Efforts are already underway to develop community-based evaluation tools, and we encourage experts to provide additional diagnostic codes that would enhance this capability for CMIP. At the same time, we encourage the community to contribute observations and reanalyses for model evaluation to the obs4MIPs and ana4MIPs archives. The intention i...

An interactive ocean surface albedo scheme (OSAv1.0): formulation and evaluation in ARPEGE-Climat (V6.1) and LMDZ (V5A)

“…This section details the evaluation of OSA at the global scale using global satellite product that are routinely used in Earth system model evaluation (Eyring et al, 2016b;Gleckler et al, 2008). We thus use OSA retrieved from CERES surface product to assess the simulated OSA by ARPEGE-Climat and LMDZ.…”

AerChemMIP: quantifying the effects of chemistry and aerosols in CMIP6

“…Note that both CCMI and AeroCom will contribute with their model evaluation experience and will feed selected observational data sources into the Earth System Model Evaluation Tool (ESMValTool; Eyring et al, 2016b;Righi et al, 2015). The ESMValTool will run -together with other evaluation tools such as the Program for Climate Model Diagnosis and Intercomparison (PCMDI) metrics package (PMP, Gleckler et al, 2016) -alongside the Earth System Grid Federation (ESGF) as soon as the output is submitted to the CMIP archive so that evaluation results can be made available at a time much faster than in CMIP5 (Eyring et al, 2016c). This will include the evaluation of chemistry and aerosols in the CMIP DECK and CMIP6 historical simulations.…”

Multivariable integrated evaluation of model performance with the vector field evaluation diagram

“…Model evaluation and intercomparison have become increasingly important, especially because a number of climate models are available at present. A total of 29 modelling groups and 60 climate models are involved in the Coupled Model Intercomparison Project Phase 5 (CMIP5) and more are expected to be included in its next phase (Eyring et al, 2016). In addition, more and more regional climate models have been used in regional model downscaling and intercomparison projects (e.g., Fu et al, 2005;van der Linden and Mitchell, 2009;Mearns et al, 2009;Giorgi and Gutowski, 2015).…”