Multimodel ensembles improve predictions of crop–environment–management interactions

Abstract

A recent innovation in assessment of climate change impact on agricultural production has been to use crop multimodel ensembles (MMEs). These studies usually find large variability between individual models but that the ensemble mean (e-mean) and median (e-median) often seem to predict quite well. However, few studies have specifically been concerned with the predictive quality of those ensemble predictors. We ask what is the predictive quality of e-mean and e-median, and how does that depend on the ensemble characteristics. Our empirical results are based on five MME studies applied to wheat, using different data sets but the same 25 crop models. We show that the ensemble predictors have quite high skill and are better than most and sometimes all individual models for most groups of environments and most response variables. Mean squared error of e-mean decreases monotonically with the size of the ensemble if models are added at random, but has a minimum at usually 2-6 models if best-fit models are added first. Our theoretical results describe the ensemble using four parameters: average bias, model effect variance, environment effect variance, and interaction variance. We show analytically that mean squared error of prediction (MSEP) of e-mean will always be smaller than MSEP averaged over models and will be less than MSEP of the best model if squared bias is less than the interaction variance. If models are added to the ensemble at random, MSEP of e-mean will decrease as the inverse of ensemble size, with a minimum equal to squared bias plus interaction variance. This minimum value is not necessarily small, and so it is important to evaluate the predictive quality of e-mean for each target population of environments. These results provide new information on the advantages of ensemble predictors, but also show their limitations.

Agricultural Development and Land Use Change in India: A Scenario Analysis of Trade‐Offs Between UN Sustainable Development Goals (SDGs)

“…Regarding the modeling of land use change processes with LandSHIFT, Göpel et al () show for a case study in Brazil how the utilized satellite‐based land‐cover products as well as the method used to estimate model parameters affect the calculated land use patterns. For addressing uncertainties in the structure of the crop and climate models, ensembles that apply multiple data sets and simulation models would be a promising route to further refine our study design (e.g., Semenov & Stratonovitch, ; Rosenzweig et al, ; Wallach et al, ). Sources of uncertainties in determining the parameters of the two empirical environmental impact models were already summarized in the previous paragraphs, and in further studies these should be systematically assessed by means of sensitivity and uncertainty analysis (Crosetto et al, ; Gao et al, ).…”

Global wheat production with 1.5 and 2.0°C above pre‐industrial warming

“…To analyze risks of the extreme low yields, we used a well‐tested multi‐model ensemble (Asseng et al, ; Ruane et al, ; Wallach et al, ) instead of individual wheat models, as the model ensemble has shown to reproduce observed yields and observed yield interannual variability. In Asseng et al (), the multi‐model ensemble median reproduced observed wheat yield under different warming treatments, with wheat‐growing season temperature ranging from 15 to 32°C, including extreme heat conditions.…”

“…An ensemble of 31 wheat crop models was used to assess climate change impacts for 60 representative wheat‐growing locations developed by the AgMIP‐Wheat team (Assenget al, ; Wallach et al, ). All models in the ensemble were calibrated for the phenology of local cultivars and used site‐specific soils and crop management.…”

“…This large climate × crop model setup enabled a robust multi‐model ensemble estimate (Martre et al, ; Wallach et al, ) as well as analysis of spatial heterogeneity (Liu et al, ) and inter‐model uncertainty. There were 11 treatments (baseline, five GCMs for 1.5 and five GCMs for 2.0 scenarios) simulated for 60 locations and 30 years (see additional detail on climate scenarios in Supporting Information Appendix S1 and in Ruane, Phillips, et al, ).…”

“…Process‐based crop simulation models, as tools to quantify the complexity of crop growth as driven by climate, soil, and management practice, have been widely used in climate change impact assessments at different spatial scales (Challinor, Martre, et al, ; Challinor, Watson, et al, ; Chenu et al, ; Ewert, Rötter, et al, ; Porter et al, ), including multi‐model ensemble approaches (Assenget al, ; Wang, Martre, et al, ; Wang, Lin, et al, ). The multi‐model ensemble approach has been proven to be a reliable method in reproducing the main effects anticipated for climate chance when simulations are compared with field experimental observations (including changes in temperature, heat events, rainfall, atmospheric CO 2 concentration [CO 2 ], and their interactions) (Assenget al, ; Wallach et al, ; Wang, Martre, et al, ; Wang, Lin, et al, ).…”