Accurate medium-range global weather forecasting with 3D neural networks

Abstract

Weather forecasting is important for science and society. At present, the most accurate forecast system is the numerical weather prediction (NWP) method, which represents atmospheric states as discretized grids and numerically solves partial differential equations that describe the transition between those states1. However, this procedure is computationally expensive. Recently, artificial-intelligence-based methods2 have shown potential in accelerating weather forecasting by orders of magnitude, but the forecast accuracy is still significantly lower than that of NWP methods. Here we introduce an artificial-intelligence-based method for accurate, medium-range global weather forecasting. We show that three-dimensional deep networks equipped with Earth-specific priors are effective at dealing with complex patterns in weather data, and that a hierarchical temporal aggregation strategy reduces accumulation errors in medium-range forecasting. Trained on 39 years of global data, our program, Pangu-Weather, obtains stronger deterministic forecast results on reanalysis data in all tested variables when compared with the world’s best NWP system, the operational integrated forecasting system of the European Centre for Medium-Range Weather Forecasts (ECMWF)3. Our method also works well with extreme weather forecasts and ensemble forecasts. When initialized with reanalysis data, the accuracy of tracking tropical cyclones is also higher than that of ECMWF-HRES.

FuXi: A cascade machine learning forecasting system for 15-day global weather forecast

“…It has been challenging for data-driven methods to compete with conventional physics-based numerical weather prediction models in weather forecasting due to the difficulty of reducing accumulation error. Recently, ML-based weather forecasting systems have witnessed significant breakthroughs, outperforming ECMWF HRES in 10-day forecasts with a temporal resolution of 6 hours and a spatial resolution of 0.25 • [16,17]. However, employing a single model proves insufficient to obtain optimal performance across various lead times.…”

“…To reduce the spatial and temporal dimensions of input and accelerate the training process, the space-time cube-embedding [21] is applied. A similar approach, patch embedding, which divides an image into N × N patches with each patch being transformed into a feature vector, was used in the Pangu-Weather model [16]. The cube embedding applies a 3-dimensional (3D) convolution layer, with a kernel and stride of 2×4×4 (equivalent to T 2 × H 4 × W 4 ), and output channels numbering C. Following cube embedding, a layer normalization (LayerNorm) [22] is utilized to improve training stability.…”

“…As a result, the Swin Transformer has shown superior performance on various benchmarks and is frequently used as a backbone architecture in many vision tasks. Additionally, many researchers have developed ML-based weather forecasting models using Swin Transformer blocks [10,12,15,16].…”

“…SwinRDM integrates Swin-RNN+, an improved version of SwinRNN that surpasses ECMWF HRES at a spatial resolution of 1.40625 • , with a diffusion-based super-resolution model that increases the resolution to 0.25 • . Pangu-Weather [16] shows its superior performance compared to ECMWF HRES in 7-days forecasts at a resolution of 0.25 • . Additionally, GraphCast [17], an autoregressive model that implements a graph neural network (GNN), outperforms HRES in 90% of the 2760 variable and lead time combinations in 10-day forecasts.…”

“…However, one limitation of direct forecasts is that distinct models must be trained for each lead time. The FourCastNet [12] [16]. They demonstrated that running the 24-hour model 7 times is better than running the 1-hour model 168 times as it significantly reduces the accumulation errors for 7-day forecasts.…”