Performance based sub-selection of CMIP6 models for impact assessments in Europe

Palmer, Tamzin; McSweeney, C.; Booth, Ben B. B.; Priestley, Matthew D. K.; Davini, Paolo; Brunner, Lukas; Borchert, Leonard; Menary, Matthew

doi:10.5194/esd-2022-31

Cited by 7 publications

(10 citation statements)

References 53 publications

(80 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While subselection can be based on performance alone (Ashfaq et al, 2022), studies that evaluate performance-based subselection tend to do so in conjunction with independence (Evans et al, 2013;Sanderson et al, 2015;Herger et al, 2018;Di Virgilio et al, 2022;Palmer et al, 2022). Evans et al (2013) succinctly demonstrated that for small subsets to reflect the spread of larger ensembles, it is more important to account for model independence (defined in the study following Bishop and Abramowitz (2013)) than for model performance.…”

Section: As Part Of the Earth Systemmentioning

confidence: 99%

“…CMIP5's interdependencies allowed for stages of subselection, first removing redundant simulations (without reducing the effective number of models), then removing poor performing simulations to improve ensemble mean mean state representation. More recently, Di Virgilio et al (2022) and Palmer et al (2022) built on these CMIP5-era strategies to support CMIP6 model subselection for CORDEX-Australasia and Europe, respectively. In Di Virgilio et al (2022), CMIP6 models, represented by an individual ensemble member, were first filtered by performance for Australian climate applications, with top and mid-tier performers further evaluated for dependencies based on the methods of Bishop and Abramowitz (2013) and Herger et al (2018).…”

Section: As Part Of the Earth Systemmentioning

confidence: 99%

“…The study then went a step further to also assess climate change signal diversity to determine whether their high performing, independent subset effectively sampled the range of Australian climatic changes in CMIP6. In Palmer et al (2022), a process-based European performance assessment for CMIP6 is presented. The study, an extension the work of McSweeney et al (2015), also incorporates a second filter based on ClimWIP's dependence definition (Brunner et al, 2020b) and notably finds that regional model selection can differ from approaches targeting global metrics such as ECS that were central to CMIP5-era EURO-CORDEX recommendations.…”

Section: As Part Of the Earth Systemmentioning

confidence: 99%

See 2 more Smart Citations

Climate model Selection by Independence, Performance, and Spread (ClimSIPS) for regional applications

Merrifield

Brunner

Lorenz

et al. 2023

Preprint

View full text Add to dashboard Cite

Abstract. As the number of models in Coupled Model Intercomparison Project (CMIP) archives increase from generation to generation, there is a pressing need for guidance on how to interpret and best use the abundance of newly available climate information. CMIP6 users seeking to draw conclusions about model agreement must contend with an "ensemble of opportunity" containing similar models that appear under different names. Those who used CMIP5 as a basis for downstream applications must filter through hundreds of new CMIP6 simulations to find several best suited to their region, season, and climate horizon of interest. Here we present methods to address both issues, model dependence and model subselection, to help users previously anchored in CMIP5 to navigate CMIP6. In Part I, we refine a definition of model dependence based on climate output, initially employed in Climate model Weighting by Independence and Performance (ClimWIP), to designate discrete model families within CMIP5/6. We show that the increased presence of model families in CMIP6 bolsters the upper mode of the ensemble's bimodal effective Equilibrium Climate Sensitivity (ECS) distribution. Accounting for the mismatch in representation between model families and individual model runs shifts the CMIP6 ECS median and 75th percentile down by 0.43 °C, achieving better alignment with CMIP5's ECS distribution. In Part II, we present a new, cost-function minimization-based approach to model subselection, Climate model Selection by Independence, Performance, and Spread (ClimSIPS), that selects sets of CMIP models based on the relative importance a user ascribes to model independence (as defined in Part I), model performance, and ensemble spread in projected climate outcome. We demonstrate ClimSIPS by selecting sets of three to five models from CMIP5/6 for European applications, evaluating the performance from the agreement with the observed mean climate, and the spread in outcome from the projected midcentury change in surface air temperature and precipitation. To accommodate different use cases, we explore two ways to represent models with multiple members in ClimSIPS, first, by ensemble mean and second, by an individual ensemble member that maximizes midcentury change diversity within CMIP overall. Because different combinations of models are selected by the cost function for different balances of independence, performance, and spread priority, we present all selected subsets in ternary contour "subselection triangles" and guide users with recommendations based on further qualitative independence, performance, and spread standards. In CMIP6, we find that recommended subsets are populated primarily by members of several model families defined in Part I due to an inverse relationship between performance and independence. In CMIP5, recommended subsets feature model combinations used in the European branch of the Coordinated Regional Downscaling Experiment (EURO-CORDEX), suggesting the independence, performance, and spread metrics used in ClimSIPS are appropriate for European applications in CMIP6 and beyond.

show abstract

Section: As Part Of the Earth Systemmentioning

confidence: 99%

Section: As Part Of the Earth Systemmentioning

confidence: 99%

Section: As Part Of the Earth Systemmentioning

confidence: 99%

See 1 more Smart Citation

Climate model Selection by Independence, Performance, and Spread (ClimSIPS) for regional applications

Merrifield

Brunner

Lorenz

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…However, it is important to note that judging climate models based solely on their ECS values may result in the removal of models that have desirable characteristics at the regional scale (e.g. Palmer et al, 2022). Additionally, keeping hot models may also be useful from an impact perspective as they may provide a clearer picture of future changes, as internal 300 variability is less likely to obscure changes.…”

Section: Resultsmentioning

confidence: 99%

“…For instance, Shiogama et al (2021) proposed a 60 subset selection method that involves screening out hot models as the first step. On the other hand, Palmer et al (2022) found that models with higher sensitivity better represent some key climatic processes over Europe. While they were unable to provide robust physical explanations for their findings, it is worth noting that at the regional scale, hot models may provide valuable information that may be more important than the global warming trend for impact modelers, adding to the complexity of selecting models for regional impact studies.…”

Section: Introduction 25mentioning

confidence: 99%

The Dilemma of Including 'Hot' Models in Climate Impact Studies: A Hydrological Study

Asenjan¹,

Brissette²,

Martel³

et al. 2023

Preprint

View full text Add to dashboard Cite

Abstract. Efficient adaptation strategies to climate change require estimating future impacts and the uncertainty surrounding this estimation. Over- or under-estimating future uncertainty may lead to maladaptation. Hydrological impact studies typically use a top-down approach in which multiple climate models are used to assess the uncertainty related to climate model structure and climate sensitivity. Despite ongoing debate, impact modelers have typically embraced the concept of "model democracy" in which each climate model is considered equally fit. The newer CMIP6 simulations, with several models showing a climate sensitivity larger than that of CMIP5 and larger than the likely range based on past climate information and understanding of planetary physics, have reignited the model democracy debate. Some have suggested that hot models be removed from impact studies to avoid skewing impact results toward unlikely futures. This large-sample study looks at the impact of removing hot models on the projections of future streamflow over 3,107 North American catchments. More precisely, the variability of future projections of mean, high, and low flows is evaluated using an ensemble of 19 CMIP6 GCMs, 5 of which are deemed "hot" based on their global equilibrium climate sensitivity (ECS). The results show that the reduced ensemble of 14 climate models provides streamflow projections with reduced future variability for Canada, Alaska, the Southwest US, and along the Pacific coast. Elsewhere, the reduced ensemble has either no impact or results in increased variability of future streamflow, indicating that outlier climate models do not necessarily provide outlier projections of future impacts. These results emphasize the delicate nature of climate model selection, especially based on global fitness metrics that may not be appropriate for local and regional assessments.

show abstract

Ex-post assessment of climate and hydrological projections: reliability of CMPI6 outputs in Northern Italy

Fuso,

Bombelli,

Bocchiola

2023

Theor Appl Climatol

View full text Add to dashboard Cite

This paper presents a validation of outputs from some GCMs of the CMIP6 project when used to assess climate projection and hydrological flows at a catchment scale for the case study area of the Lombardy region (Northern Italy). The modeling chain consists of (i) a choice of climatic scenarios from 10 GCMs of the CMIP6, (ii) the application of a stochastic downscaling procedure to make projections usable at the local scale, and (iii) the use of a semi-distributed physically based hydrological model Poli-Hydro for the generation of hydrological scenarios. Data on observed precipitation and temperature were collected from automatic weather stations, and the hydrological budget of four target catchments within the study area was assessed using Poli-Hydro. An ex-post (back-casting) analysis was performed upon the control data series from the GCMs by comparing statistics of relevant climate variables and model-simulated discharges against observed counterparts during the historical period 2002–2014. Then, during 2015–2021, the goodness of projections was assessed using confidence intervals. Our results show that the accuracy of GCMs in representing regional climate is not always reflected in a credible evaluation of local hydrology. The validation of climate patterns provides somewhat poor results; thus, the interaction among climate and hydrology needs to be explored carefully to warrant the credibility of hydrological scenarios. Overall, the spatial and temporal consistency of GCM projections, as explored here climatically and hydrologically, provides a clue about their dependability for basin scale management.

show abstract

Performance based sub-selection of CMIP6 models for impact assessments in Europe

Cited by 7 publications

References 53 publications

Climate model Selection by Independence, Performance, and Spread (ClimSIPS) for regional applications

Climate model Selection by Independence, Performance, and Spread (ClimSIPS) for regional applications

The Dilemma of Including 'Hot' Models in Climate Impact Studies: A Hydrological Study

Ex-post assessment of climate and hydrological projections: reliability of CMPI6 outputs in Northern Italy

Contact Info

Product

Resources

About