Within the Copernicus Climate Change Service (C3S), ECMWF is producing the ERA5 reanalysis which, once completed, will embody a detailed record of the global atmosphere, land surface and ocean waves from 1950 onwards. This new reanalysis replaces the ERA-Interim reanalysis (spanning 1979 onwards) which was started in 2006. ERA5 is based on the Integrated Forecasting System (IFS) Cy41r2 which was operational in 2016. ERA5 thus benefits from a decade of developments in model physics, core dynamics and data assimilation. In addition to a significantly enhanced horizontal resolution of 31 km, compared to 80 km for ERA-Interim, ERA5 has hourly output throughout, and an uncertainty estimate from an ensemble (3-hourly at half the horizontal resolution). This paper describes the general setup of ERA5, as well as a basic evaluation of characteristics and performance, with a focus on the dataset from 1979 onwards which is currently publicly available. Re-forecasts from ERA5 analyses show a gain of up to one day in skill with respect to ERA-Interim. Comparison with radiosonde and PILOT data prior to assimilation shows an improved fit for temperature, wind and humidity in the troposphere, but not the stratosphere. A comparison with independent buoy data shows a much improved fit for ocean wave height. The uncertainty estimate reflects the evolution of the observing systems used in ERA5. The enhanced temporal and spatial resolution allows for a detailed evolution of weather systems. For precipitation, global-mean correlation with monthly-mean GPCP data is increased from 67% This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

The ECMWF twentieth century reanalysis (ERA-20C; 1900–2010) assimilates surface pressure and marine wind observations. The reanalysis is single-member, and the background errors are spatiotemporally varying, derived from an ensemble. The atmospheric general circulation model uses the same configuration as the control member of the ERA-20CM ensemble, forced by observationally based analyses of sea surface temperature, sea ice cover, atmospheric composition changes, and solar forcing. The resulting climate trend estimations resemble ERA-20CM for temperature and the water cycle. The ERA-20C water cycle features stable precipitation minus evaporation global averages and no spurious jumps or trends. The assimilation of observations adds realism on synoptic time scales as compared to ERA-20CM in regions that are sufficiently well observed. Comparing to nighttime ship observations, ERA-20C air temperatures are 1 K colder. Generally, the synoptic quality of the product and the agreement in terms of climate indices with other products improve with the availability of observations. The MJO mean amplitude in ERA-20C is larger than in 20CR version 2c throughout the century, and in agreement with other reanalyses such as JRA-55. A novelty in ERA-20C is the availability of observation feedback information. As shown, this information can help assess the product’s quality on selected time scales and regions.

CERA-20C is a coupled reanalysis of the twentieth century which aims to reconstruct the past weather and climate of the Earth system including the atmosphere, ocean, land, ocean waves, and sea ice. This reanalysis is based on the CERA coupled atmosphere-ocean assimilation system developed at ECMWF. CERA-20C provides a 10 member ensemble of reanalyses to account for errors in the observational record as well as model error. It benefited from the prior experience of the retrospective atmospheric analysis ERA-20C. The dynamical model and the data assimilation systems initially developed for NWP had been modified to take into account the evolution of the radiative forcing and the observing system. To limit the impact of changes in the observing system throughout the century, only conventional surface observations have been used in the atmosphere. CERA-20C improves the specification of the background and the observation errors, two key elements to ensure a consistent weighting of the uncertainties across geophysical variables, space, and time. The quality of CERA-20C has been evaluated against other centennial reanalyses and independent observations. Although CERA-20C inherits some limitations of ERA-20C to represent correctly the tropical cyclones in the first part of the century, it shows significant improvements in the troposphere, compared to ERA-20C and 20CRv2c (the twentieth century reanalysis produced by NOAA/CIRES). A preliminary study of the climate variability in CERA-20C has been carried out. CERA-20C improves on the representation of atmosphere-ocean heat fluxes and mean sea level pressure compared to previous uncoupled ocean and atmospheric historical reanalyses performed at ECMWF.

A coupled data assimilation system has been developed at the European Centre for Medium‐Range Weather Forecasts (ECMWF), which is intended to be used for the production of global reanalyses of the recent climate. The system assimilates a wide variety of ocean and atmospheric observations and produces ocean–atmosphere analyses with a coupled model. Employing the coupled‐model constraint in the analysis implies that assimilation of an ocean observation has immediate impact on the atmospheric state estimate and, conversely, assimilation of an atmospheric observation affects the ocean state. This covariance between atmosphere and ocean induced by the analysis method is illustrated with simple numerical experiments. Realistic data assimilation experiments based on the global observing system are then used to assess the quality of the assimilation method. Comparison with an uncoupled system shows a mostly neutral impact overall, with slightly improved temperature estimates in the upper ocean and lower atmosphere. These preliminary results are considered of interest for the ongoing community efforts focusing on coupled data assimilations.

Model error is one of the main obstacles to improved accuracy and reliability in numerical weather prediction (NWP) and climate prediction conducted with state-of-the-art, comprehensive high-resolution general circulation models. In a data assimilation framework, recent advances in the context of weak-constraint 4D-Var have shown that it is possible to estimate and correct for a large fraction of systematic model error which develops in the stratosphere over short forecast ranges. The recent explosion of interest in machine learning/deep learning technologies has been driven by their remarkable success in disparate application areas. This raises the question of whether model error estimation and correction in operational NWP and climate prediction can also benefit from these techniques. In this work, we aim to start to give an answer to this question. Specifically, we show that artificial neural networks (ANNs) can reproduce the main results obtained with weak-constraint 4D-Var in the operational configuration of the IFS model of the European Centre for Medium-Range Weather Forecasts (ECMWF). We show that the use of ANN models inside the weak-constraint 4D-Var framework has the potential to extend the applicability of the weak-constraint methodology for model error correction to the whole atmospheric column. Finally, we discuss the potential and limitations of the machine learning/deep learning technologies in the core NWP tasks. In particular, we reconsider the fundamental constraints of a purely data-driven approach to forecasting and provide a view on how to best integrate machine learning technologies within current data assimilation and forecasting methods. Plain Language Summary Model error is one of the main obstacles to improved accuracy and reliability in current numerical weather prediction and in climate prediction. Recent advances in data assimilation at the European Centre for Medium-Range Weather Forecasts (ECMWF) indicate that it is possible to estimate and correct for a large fraction of systematic model error in the stratosphere. The question we address here is whether machine learning techniques can be used alone and in conjunction with standard data assimilation methods to improve on those results. We show that it is indeed possible to extend current data assimilation capabilities in operational state-of-the-art forecast systems using machine learning tools, and we discuss the potential and limitations of future applications of these ideas to other core NWP tasks.

The idea of using machine learning (ML) methods to reconstruct the dynamics of a system is the topic of recent studies in the geosciences, in which the key output is a surrogate model meant to emulate the dynamical model. In order to treat sparse and noisy observations in a rigorous way, ML can be combined with data assimilation (DA). This yields a class of iterative methods in which, at each iteration, a DA step assimilates the observations and alternates with a ML step to learn the underlying dynamics of the DA analysis. In this article, we propose to use this method to correct the error of an existing, knowledge-based model. In practice, the resulting surrogate model is a hybrid model between the original (knowledge-based) model and the ML model. We demonstrate the feasibility of the method numerically using a two-layer, two-dimensional, quasi-geostrophic channel model. Model error is introduced by the means of perturbed parameters. The DA step is performed using the strong-constraint 4D-Var algorithm, while the ML step is performed using deep learning tools. The ML models are able to learn a substantial part of the model error and the resulting hybrid surrogate models produce better short-to mid-range forecasts. Furthermore, using the hybrid surrogate models for DA yields a significantly better analysis than using the original model.

The standard, strong‐constraint formulation of 4D‐Var is designed to correct for random, zero‐mean errors from the model forecast and the observations. However, significant systematic errors are generated by the forecast models used in global numerical weather prediction (NWP), and the Integrated Forecast System (IFS) model of the European Centre for Medium‐Range Weather Forecasts (ECMWF) is no exception. To deal with this type of error, a modification of the standard 4D‐Var algorithm, weak‐constraint 4D‐Var, has long been proposed and was implemented in operations at ECMWF in 2009. In the original implementation, the model error corrections were only active in the upper stratosphere, as a full application of the algorithm would lead to unacceptable degradation of forecast scores. Thus the impact of weak‐constraint 4D‐Var on ECMWF analyses and forecasts has always been small heretofore.In a companion article, it was shown through theoretical considerations and experiments with a simplified model that weak‐constraint 4D‐Var can correct accurately for model errors and initial‐state errors only when background and model errors have different spatial scales and when the observing system has a component, sufficient in number, that is unbiased and spatially homogeneous. This article presents new diagnostics based on radio occultation (RO) temperature retrievals and shows how those assumptions are verified in the IFS model. A new model‐error covariance matrix is derived and reduces the analysis and first‐guess temperature bias in the stratosphere by up to 50%. We also present how the model error tendencies diagnosed by weak‐constraint 4D‐Var can be used to reduce mean errors in medium‐range forecasts and discuss how they can also be applied in a reanalysis context to improve the fidelity and continuity of climate reconstructions.

Current methods for initializing coupled atmosphere–ocean forecasts often rely on the use of separate atmosphere and ocean analyses, the combination of which can leave the coupled system imbalanced at the beginning of the forecast, potentially accelerating the development of errors. Using a series of experiments with the European Centre for Medium-Range Weather Forecasts coupled system, the magnitude and extent of these so-called initialization shocks is quantified, and their impact on forecast skill measured. It is found that forecasts initialized by separate oceanic and atmospheric analyses do exhibit initialization shocks in lower atmospheric temperature, when compared to forecasts initialized using a coupled data assimilation method. These shocks result in as much as a doubling of root-mean-square error on the first day of the forecast in some regions, and in increases that are sustained for the duration of the 10-day forecasts performed here. However, the impacts of this choice of initialization on forecast skill, assessed using independent datasets, were found to be negligible, at least over the limited period studied. Larger initialization shocks are found to follow a change in either the atmosphere or ocean model component between the analysis and forecast phases: changes in the ocean component can lead to sea surface temperature shocks of more than 0.5 K in some equatorial regions during the first day of the forecast. Implications for the development of coupled forecast systems, particularly with respect to coupled data assimilation methods, are discussed.