Characterizing the performance of ecosystem models across time scales: A spectral analysis of the North American Carbon Program site-level synthesis

Abstract

covariance towers as part of the North American Carbon Program's site-level intercomparison. This study expands upon previous single-site and single-model analyses to determine what patterns of model error are consistent across a diverse range of models and sites. To assess the significance of model error at different time scales, a novel Monte Carlo approach was developed to incorporate flux observation error. Failing to account for observation error leads to a misidentification of the time scales that dominate model error. These analyses show that model error (1) is largest at the annual and 20-120 day scales, (2) has a clear peak at the diurnal scale, and (3) shows large variability among models in the 2-20 day scales. Errors at the annual scale were consistent across time, diurnal errors were predominantly during the growing season, and intermediate-scale errors were largely event driven. Breaking spectra into discrete temporal bands revealed a significant model-by-band effect but also a nonsignificant model-by-site effect, which together suggest that individual models show consistency in their error patterns. Differences among models were related to model time step, soil hydrology, and the representation of photosynthesis and phenology but not the soil carbon or nitrogen cycles. These factors had the greatest impact on diurnal errors, were less important at annual scales, and had the least impact at intermediate time scales.

Carbon and energy fluxes in cropland ecosystems: a model-data comparison

“…Similar to some previous analyses done under the NACP Site Synthesis (Schwalm et al 2010; 123 Dietz et al 2012;Schaefer et al 2008;Stoy et al 2013), the output from those models that simulated agricultural sites was analyzed at a monthly time scale, to account for the variation in the temporal resolution among the models (hourly, daily, weekly, and monthly). Taylor diagrams based on modeled flux standard deviation (STD), root mean square error (RMSE), and correlations with the observed data were used to evaluate the model output against the observed, non-gap filled fluxes of carbon (NEE), and energy (LE, and H).…”

“…For instance, Schwalm et al (2010) analyzed the CO 2 exchange simulated by 22 terrestrial biosphere models at 44 eddy covariance flux towers in North America and found that few models simulating different biomes and sites, the mean model ensemble, and a model that used data assimilation for parameter optimization showed high consistency with observations. Dietz et al (2012) used spectral analyses to determine the performance of 21 ecosystem models at multiple time scales considering 9 eddy covariance flux tower sites; this study found that the model performance was related to model time step, soil hydrology, and the representation of photosynthesis and phenology in the models. Stoy et al (2013) used wavelet coherence to analyze the model performance of 20 ecosystem models at 10 different eddy covariance research sites in simulating NEE at different time scales and identified the need for better parameterization and mechanistic improvement of models for more accurate predictions.…”

Terrestrial biosphere models need better representation of vegetation phenology: results from the North American Carbon Program Site Synthesis

“…(We do not intend for this study to be considered a comprehensive analysis of all aspects of model performance; complementary NACP efforts include work by Schwalm et al (2010) and Dietze et al (2011), and work in preparation by K. Schaefer et al, T. Keenan et al, P. Stoy et al, and B. Raczka et al.) The five deciduous broadleaf forest (DBF) and five evergreen needleleaf forest (ENF) sites selected for the analysis are all members of either the AmeriFlux or Fluxnet-Canada networks.…”

Combining multiple statistical methods to evaluate the performance of process-based vegetation models across three forest stands

“…Firstly, the coefficient of determination (R 2 ), secondly, R 2 multiplied by the slope of the regression line between simulations and observations (bR 2 ), allowing to account for the systematic discrepancy in the magnitude of two signals as well as for the proportion of variance in the observations pre- in variance amplitude, respectively (Dietze et al 2011). The day of the year (DOY) averages of Res cn over the whole study period were calculated to check for systematic asynchronies between simulations and observations, and were compared with the intra-annual NEE evolution represented in the same way.…”

“…As residual analysis (RA) examines model errors as a function of simulated or observed data and of environmental drivers, it may reveal potential model shortcomings (Medlyn et al 2005). More complex time series analysis techniques including wavelet analysis (WA; Stoy et al 2005;Dietze et al 2011) and singular spectrum analysis (SSA; Mahecha et al 2007;Mahecha et al 2010;Wang et al 2012) effectively provide insight into the model fit at different time scales. In this contribution, we used two DGVMs and one forest stand-scale model, to simulate NEE.…”