[1] Objective measures of climate model performance are proposed and used to assess simulations of the 20th century, which are available from the Coupled Model Intercomparison Project (CMIP3) archive. The primary focus of this analysis is on the climatology of atmospheric fields. For each variable considered, the models are ranked according to a measure of relative error. Based on an average of the relative errors over all fields considered, some models appear to perform substantially better than others. Forming a single index of model performance, however, can be misleading in that it hides a more complex picture of the relative merits of different models. This is demonstrated by examining individual variables and showing that the relative ranking of models varies considerably from one variable to the next. A remarkable exception to this finding is that the so-called ''mean model'' consistently outperforms all other models in nearly every respect. The usefulness, limitations and robustness of the metrics defined here are evaluated 1) by examining whether the information provided by each metric is correlated in any way with the others, and 2) by determining how sensitive the metrics are to such factors as observational uncertainty, spatial scale, and the domain considered (e.g., tropics versus extra-tropics). An index that gauges the fidelity of model variability on interannual time-scales is found to be only weakly correlated with an index of the mean climate performance. This illustrates the importance of evaluating a broad spectrum of climate processes and phenomena since accurate simulation of one aspect of climate does not guarantee accurate representation of other aspects. Once a broad suite of metrics has been developed to characterize model performance it may become possible to identify optimal subsets for various applications.

The Atmospheric Model Intercomparison Project (AMIP), initiated in 1989 under the auspices of the World Climate Research Programme, undertook the systematic validation, diagnosis, and intercomparison of the performance of atmospheric general circulation models. For this purpose all models were required to simulate the evolution of the climate during the decade 1979-88, subject to the observed monthly average temperature and sea ice and a common prescribed atmospheric C02 concentration and solar constant. By 1995, 31 modeling groups, representing virtually the entire international atmospheric modeling community, had contributed the required standard output of the monthly means of selected statistics. These data have been analyzed by the participating modeling groups, by the Program for Climate Model Diagnosis and Intercomparison, and by the more than two dozen AMIP diagnostic subprojects that have been established to examine specific aspects of the models' performance. Here the analysis and validation of the AMIP results as a whole are summarized in order to document the overall performance of atmospheric general circulation-climate models as of the early 1990s. The infrastructure and plans for continuation of the AMIP project are also reported on.Although there are apparent model outliers in each simulated variable examined, validation of the AMIP models' ensemble mean shows that the average large-scale seasonal distributions of pressure, temperature, and circulation are reasonably close to what are believed to be the best observational estimates available. The large-scale structure of the ensemble mean precipitation and ocean surface heat flux also resemble the observed estimates but show particularly large intermodel differences in low latitudes. The total cloudiness, on the other hand, is rather poorly simulated, especially in the Southern Hemisphere. The models' simulation of the seasonal cycle (as represented by the amplitude and phase of the first annual harmonic of sea level pressure) closely resembles the observed variation in almost all regions. The ensemble's simulation of the interannual variability of sea level pressure in the tropical Pacific is reasonably close to that observed (except for its underestimate of the amplitude of major El Ninos), while the interannual variability is less well simulated in midlatitudes. When analyzed in terms of the variability of the evolution of their combined spacetime patterns in comparison to observations, the AMIP models are seen to exhibit a wide range of accuracy, with no single model performing best in all respects considered.Analysis of the subset of the original AMIP models for which revised versions have subsequently been used to revisit the experiment shows a substantial reduction of the models' systematic errors in simulating cloudiness but only a slight reduction of the mean seasonal errors of most other variables. In order to understand better the nature of these errors and to accelerate the rate of model improvement, an expanded and continuing project (...

26. We chose these values of the target factors to produce our final results because we have concluded that they are the most likely to be free of errors. They are calculated from oceanic observations to reduce errors from uncorrected diurnal variations, and we use unweighted MSU channel 2 data (T2 in SOM) to avoid additional noise due to the differencing procedure used to calculate TLT. The values of the intersatellite offsets needed to be recalculated to remove obvious intersatellite differences. In the supporting online material, we discuss the impact of using different data subsets to determine the target factors. This information is used to help determine the structural uncertainty. 27. We obtain this estimate of the tropical TLT trend when we recalculate the intersatellite offsets to optimize them for tropical data. If this reoptimization is not performed, as it is not in producing maps such as those shown in Fig. 3, we obtain a smaller trend value of 0. The month-to-month variability of tropical temperatures is larger in the troposphere than at Earth's surface. This amplification behavior is similar in a range of observations and climate model simulations and is consistent with basic theory. On multidecadal time scales, tropospheric amplification of surface warming is a robust feature of model simulations, but it occurs in only one observational data set. Other observations show weak, or even negative, amplification. These results suggest either that different physical mechanisms control amplification processes on monthly and decadal time scales, and models fail to capture such behavior; or (more plausibly) that residual errors in several observational data sets used here affect their representation of long-term trends.Tropospheric warming is a robust feature of climate model simulations that include historical increases in greenhouse gases (1-3). Maximum warming is predicted to occur in the middle and upper tropical troposphere. Atmospheric temperature measurements from radiosondes also show warming of the tropical troposphere since the early 1960s (4-7), consistent with model results (8). The observed tropical warming is partly due to a step-like change in the late 1970s (5, 6). Considerable attention has focused on the shorter record of satellite-based atmospheric temperature measurements (1979 to present). In both models and observations, the tropical surface warms over this period. Simulated surface warming is amplified in the tropical troposphere, corresponding to a decrease in lapse rate (2,3,9). In contrast, a number of radiosonde and satellite data sets suggest that the tropical troposphere has warmed less than the surface, or even cooled, which would correspond to an increase in lapse rate (4)(5)(6)(7)(8)(9)(10)(11)(12).This discrepancy may be an artifact of residual inhomogeneities in the observations (13)(14)(15)(16)(17)(18)(19). Creating homogeneous climate records requires the identification and removal of nonclimatic influences from data that were primarily collected for weather forecasting...