Association of parameter, software, and hardware variation with large-scale behavior across 57,000 climate models

Abstract

In complex spatial models, as used to predict the climate response to greenhouse gas emissions, parameter variation within plausible bounds has major effects on model behavior of interest. Here, we present an unprecedentedly large ensemble of >57,000 climate model runs in which 10 parameters, initial conditions, hardware, and software used to run the model all have been varied. We relate information about the model runs to large-scale model behavior (equilibrium sensitivity of global mean temperature to a doubling of carbon dioxide). We demonstrate that effects of parameter, hardware, and software variation are detectable, complex, and interacting. However, we find most of the effects of parameter variation are caused by a small subset of parameters. Notably, the entrainment coefficient in clouds is associated with 30% of the variation seen in climate sensitivity, although both low and high values can give high climate sensitivity. We demonstrate that the effect of hardware and software is small relative to the effect of parameter variation and, over the wide range of systems tested, may be treated as equivalent to that caused by changes in initial conditions. We discuss the significance of these results in relation to the design and interpretation of climate modeling experiments and large-scale modeling more generally.classification and regression trees ͉ climate change ͉ distributed computing ͉ general circulation models ͉ sensitivity analysis S imulation with complex mechanistic spatial models is central to science from the level of molecules (1) via biological systems (2, 3) to global climate (4). The objective typically is a mechanistically based prediction of system-level behavior. However, both through incomplete knowledge of the system simulated and the approximations required to make such models tractable, the ''true'' or ''optimal'' values of some model parameters necessarily will be uncertain. A limiting factor in such simulations is the availability of computational resources. Thus, combinations of plausible parameter values rarely are tested, leaving the dependence of conclusions on the particular parameters chosen unknown.Observations of the modeled system are vital for model verification and analysis, e.g., turning model output into probabilistic predictions of real-world system behavior (5-7). However, typically, few observations are available relative to the complexity of the model. There also may be little true replicate data available. For instance, there can be only one observational time series for global climate. Thus, if the same observations are used to fit parameter values, there is a severe risk of overfitting, gaining limited verisimilitude at the cost of the mechanistic insight and predictive ability for which the model originally was designed.To avoid fitting problems, parameter estimates must be refined directly. In some biological systems, direct and simultaneous measurement of large numbers of system parameters (e.g., protein binding or catalytic constants) soon may be possible. I...

On the role of the planetary boundary layer depth in the climate system

“…The PBL mass budget leads to the direct proportionality between the PBL depth, h, and the entrainment rate (Medeiros et al, 2005). Knight et al (2007) reported the highest climate sensitivity for the lowest A, found in the shallowest PBL. The reciprocal relationship between the climate sensitivity and the PBL depth suggests larger climate variability in stably stratified boundary layers (Esau, 2008;Zilitinkevich and Esau, 2010), where h is small, typically less than 200 m in nocturnal PBL as to be compared to its daytime counterpart with h > 1000 m (e.g.…”

“…Knight et al, 2007), which excurse beyond the current narrow focus on PBL parameterizations in climate models. Using a classification and regression tree approach, Knight et al (2007) demonstrated with 57 067 climate model HadAM3 runs that 80% of variation in climate sensitivity to 2 × CO 2 is associated with variation of a small subset of parameters mostly related to the convection proCorrespondence to: I. Esau (igore@nersc.no) cesses.…”

Considerations for parameter optimization and sensitivity in climate models

“…climate model optimization | metamodeling | precipitation bias and sensitivity I nterest in systematic parameter sensitivity and optimization has been developing both in the context of global average climate sensitivity associated with increased greenhouse gases and the effort to improve the model climatology (1)(2)(3)(4)(5)(6)(7). Some of this work has focused on variations with parameter of a climate sensitivity defined by the change of global average surface temperature under doubled CO 2 , some on optimizing the simulation of current climate features by tuning parameter values.…”

Facing future climate change: is the past relevant?

“…Uncertainties in model predictions can, and do, arise from structural simplifications such as these (Smith 2002;Stainforth et al 2007). In highly complex global climate models, uncertainties also arise from parametrizations ('knob settings') and inherent chaotic behaviour, both of which may be tackled without specific geological hindsight (Stainforth et al 2005(Stainforth et al , 2007Knight et al 2007). However, if we are to assess the degree to which model truncations may result in qualitatively impaired behaviour, access to independent historical information is imperative.…”