• Benchmarks with strong baselines are a key ingredient for rapid progress on a problem. • Here, we define a benchmark for data-driven global, medium-range weather prediction. • The data is processed for convenient use in machine learning models and a quickstart guide is provided.

We develop elementary weather prediction models using deep convolutional neural networks (CNNs) trained on past weather data to forecast one or two fundamental meteorological fields on a Northern Hemisphere grid with no explicit knowledge about physical processes. At forecast lead times up to 3 days, CNNs trained to predict only 500‐hPa geopotential height easily outperform persistence, climatology, and the dynamics‐based barotropic vorticity model, but do not beat an operational full‐physics weather prediction model. These CNNs are capable of forecasting significant changes in the intensity of weather systems, which is notable because this is beyond the capability of the fundamental dynamical equation that relies solely on 500‐hPa data, the barotropic vorticity equation. Modest improvements to the CNN forecasts can be made by adding 700‐ to 300‐hPa thickness to the input data. Our best performing CNN does a good job of capturing the climatology and annual variability of 500‐hPa heights and is capable of forecasting realistic atmospheric states at lead times of 14 days. Although our simple models do not perform better than an operational weather model, machine learning warrants further exploration as a weather forecasting tool; in particular, the potential efficiency of CNNs might make them attractive for ensemble forecasting.

We present a significantly improved data‐driven global weather forecasting framework using a deep convolutional neural network (CNN) to forecast several basic atmospheric variables on a global grid. New developments in this framework include an off‐line volume‐conservative mapping to a cubed‐sphere grid, improvements to the CNN architecture and the minimization of the loss function over multiple steps in a prediction sequence. The cubed‐sphere remapping minimizes the distortion on the cube faces on which convolution operations are performed and provides natural boundary conditions for padding in the CNN. Our improved model produces weather forecasts that are indefinitely stable and produce realistic weather patterns at lead times of several weeks and longer. For short‐ to medium‐range forecasting, our model significantly outperforms persistence, climatology, and a coarse‐resolution dynamical numerical weather prediction (NWP) model. Unsurprisingly, our forecasts are worse than those from a high‐resolution state‐of‐the‐art operational NWP system. Our data‐driven model is able to learn to forecast complex surface temperature patterns from few input atmospheric state variables. On annual time scales, our model produces a realistic seasonal cycle driven solely by the prescribed variation in top‐of‐atmosphere solar forcing. Although it currently does not compete with operational weather forecasting models, our data‐driven CNN executes much faster than those models, suggesting that machine learning could prove to be a valuable tool for large‐ensemble forecasting.

One important limitation on the accuracy of weather forecasts is imposed by unavoidable errors in the specification of the atmosphere’s initial state. Much theoretical concern has been focused on the limits to predictability imposed by small-scale errors, potentially even those on the scale of a butterfly. Very modest errors at much larger scales may nevertheless pose a more important practical limitation. We demonstrate the importance of large-scale uncertainty by analyzing ensembles of idealized squall-line simulations. Our results imply that minimizing initial errors on scales around 100 km is more likely to extend the accuracy of forecasts at lead times longer than 3–4 h than efforts to minimize initial errors on much smaller scales. These simulations also demonstrate that squall lines, triggered in a horizontally homogeneous environment with no initial background circulations, can generate a background mesoscale kinetic energy spectrum roughly similar to that observed in the atmosphere.

An ensemble forecast system is developed using convolution neural networks (CNNs) to generate data-driven global forecasts.• Only 3 seconds are required to compute a large 320-member ensemble of skillful 10 6-week sub-seasonal predictions. 11• Shorter lead time forecasts also show skill, including a single deterministic 4-day 12 forecast for Hurricane Irma.

Recent work has suggested that modest initial relative errors on scales of O(100) km in a numerical weather forecast may exert more control on the predictability of mesoscale convective systems at lead times beyond about 5 h than 100% relative errors at smaller scales. Using an idealized model, the predictability of deep convection organized by several different profiles of environmental vertical wind shear is investigated as a function of the horizontal scale and amplitude of initial errors in the low-level moisture field. Small- and large-scale initial errors are found to have virtually identical impacts on predictability at lead times of 4–5 h for all wind shear profiles. Both small- and large-scale errors grow primarily up in amplitude at all scales rather than through an upscale cascade between adjacent scales. Reducing the amplitude of the initial errors improves predictability lead times, but this improvement diminishes with further reductions in the error amplitude, suggesting a limit to the intrinsic predictability in these simulations of slightly more than 6 h at scales less than 20 km. Additionally, all the simulated convective systems produce a k−5/3 spectrum of kinetic energy, providing evidence of the importance of the unbalanced, divergent gravity wave component of the flow produced by thunderstorms in generating the observed atmospheric kinetic energy spectrum.

Spectra are often computed from gridded data to determine the horizontal-scale dependence of quantities such as kinetic energy, vertical velocity, or perturbation potential temperature. This paper discusses several important considerations for the practical computation of such spectra. To ensure that the sum of the spectral energy densities in wavenumber space matches the sum of the energies in the physical domain (the discrete Parseval relation), the constant coefficient multiplying the spectral energy density must properly account for the way the discrete Fourier transform pair is normalized. The normalization factor appropriate of many older FORTRAN-based fast Fourier transforms (FFTs) differs from that in Matlab and Python’s numpy.fft, and as a consequence, the correct scaling factor for the kinetic energy (KE) spectral density differs between one-dimensional FFTs computed using these two approaches by a factor equal to the square of the number of physical grid points. A common algorithm used to compute two-dimensional spectra as a function of the total-wavenumber magnitude sums the contributions from all pairs of x- and y-component wavenumbers whose vector magnitude lies with a series of bins. This approach introduces systematic short-wavelength noise, which can be largely eliminated though a simple multiplicative correction. One- and two-dimensional spectra will differ by a constant if computed for flows in which the KE spectral density decreases as a function of the wavenumber to some negative power. This constant is evaluated and the extension of theoretical results to numerically computed FFTs is examined.

We present a significantly improved data-driven global weather forecasting framework using a deep convolutional neural network (CNN) to forecast several basic atmospheric variables on a global grid. New developments in this framework include an off-line volume-conservative mapping to a cubed-sphere grid, improvements to the CNN architecture and the minimization of the loss function over multiple steps in a prediction sequence. The cubed-sphere remapping minimizes the distortion on the cube faces on which convolution operations are performed and provides natural boundary conditions for padding in the CNN. Our improved model produces weather forecasts that are indefinitely stable and produce realistic weather patterns at lead times of several weeks and longer. For short-to medium-range forecasting, our model significantly outperforms persistence, climatology, and a coarse-resolution dynamical numerical weather prediction (NWP) model. Unsurprisingly, our forecasts are worse than those from a high-resolution state-of-the-art operational NWP system. Our data-driven model is able to learn to forecast complex surface temperature patterns from few input atmospheric state variables. On annual time scales, our model produces a realistic seasonal cycle driven solely by the prescribed variation in top-of-atmosphere solar forcing. Although it currently does not compete with operational weather forecasting models, our data-driven CNN executes much faster than those models, suggesting that machine learning could prove to be a valuable tool for large-ensemble forecasting.Plain Language Summary Recent work has begun to explore building global weather prediction models using only machine learning techniques trained on large amounts of atmospheric data. We develop a vastly improved machine learning algorithm capable of operating like traditional weather models and predicting several fundamental atmospheric variables, including near-surface temperature. While our model does not yet compete with the state-of-the-art in numerical weather prediction, it computes realistic forecasts that perform well and execute extremely quickly, offering a potential avenue for future developments in probabilistic weather forecasting.