Selecting a climate model subset to optimise key ensemble properties

Abstract

Abstract. End users studying impacts and risks caused by human-induced climate change are often presented with large multi-model ensembles of climate projections whose composition and size are arbitrarily determined. An efficient and versatile method that finds a subset which maintains certain key properties from the full ensemble is needed, but very little work has been done in this area. Therefore, users typically make their own somewhat subjective subset choices and commonly use the equally weighted model mean as a best estimate. However, different climate model simulations cannot necessarily be regarded as independent estimates due to the presence of duplicated code and shared development history.Here, we present an efficient and flexible tool that makes better use of the ensemble as a whole by finding a subset with improved mean performance compared to the multi-model mean while at the same time maintaining the spread and addressing the problem of model interdependence. Out-of-sample skill and reliability are demonstrated using model-as-truth experiments. This approach is illustrated with one set of optimisation criteria but we also highlight the flexibility of cost functions, depending on the focus of different users. The technique is useful for a range of applications that, for example, minimise present-day bias to obtain an accurate ensemble mean, reduce dependence in ensemble spread, maximise future spread, ensure good performance of individual models in an ensemble, reduce the ensemble size while maintaining important ensemble characteristics, or optimise several of these at the same time. As in any calibration exercise, the final ensemble is sensitive to the metric, observational product, and pre-processing steps used.

Prospects and Caveats of Weighting Climate Models for Summer Maximum Temperature Projections Over North America

“…This needs to be reconsidered given that multimodel ensembles contain near replicates of the same model that are not independent anymore Knutti et al, 2013;Masson & Knutti, 2011). Several methods have been proposed to take into account performance and/or interdependence when calculating multimodel averages (e.g., Abramowitz & Bishop, 2015;Abramowitz et al, 2008;Herger et al, 2018;Karpechko et al, 2013;Sanderson et al, 2015aSanderson et al, , 2015bTebaldi et al, 2006;Waugh & Eyring, 2008). Most of these approaches are rather complex, and so far no agreement was reached about which method should be used in general, how model dependence should be defined, and how it could best be accounted for.…”

Calibrating Climate Model Ensembles for Assessing Extremes in a Changing Climate

“…Since evaluating response bias to long-term changes in radiative forcing using multiple observational products is not common practice in the event attribution community, we suggest that the long-term response to forcing be evaluated and should be part of the optimization process if this characteristic of the raw model output is deemed unfit for purpose (in addition to distribution properties). The difficulty here however is that the nature of the long-term temperature response in-sample does not seem to persist out-of-sample (see, e.g., Figure 4 in Herger et al (2018)). Knutti and Hegerl (2008) showed that it is possible for two climate models to very closely track historical warming trends but behave completely different thereafter (their Figure 4).…”

Similarities within a multi-model ensemble: functional data analysis framework

“…The results of multivariate evaluation of the similarities and relationships within the multi-model ensemble could be a basis for selection of representative models to be used in impact studies. Previously proposed procedures, such as in Mendlik and Gobiet (2016) or Herger et al (2018), could be modified to use the FDA similarities introduced here. 30…”