Spatial and Interannual Variability of the Reliability of Ensemble-Based Probabilistic Forecasts: Consequences for Calibration

Atger, Frédéric

doi:10.1175//1520-0493(2003)131<1509:saivot>2.0.co;2

Cited by 81 publications

(60 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the limit of an infinite number of ensemble members the perfect ensemble will be perfectly reliable. The reliability of an ensemble can be improved through calibration based on statistics of past ensemble forecasts for the same season and region [21,22].…”

Section: Predicting Uncertaintymentioning

confidence: 99%

Ensemble forecasting

Leutbecher

Palmer

2008

Journal of Computational Physics

682

556

View full text Add to dashboard Cite

Section: Predicting Uncertaintymentioning

confidence: 99%

Ensemble forecasting

Leutbecher

Palmer

2008

Journal of Computational Physics

682

556

View full text Add to dashboard Cite

“…However, it is now common practice to calculate the components by stratifying over bins of probabilities such as those used to produce reliability diagrams (e.g., Atger 2003). This widespread usage has even led to the misconception that "estimating reliability and resolution requires a categorization of probabilistic forecasts" (Atger 2004, p. 628).…”

Section: Introductionmentioning

confidence: 99%

“…First, it acts as a crude form of smoothing thereby making the conditional means less uncertain and the reliability curve less noisy (Atger 2003). Second, larger bins can avoid sparseness problems that can occur when probabilities are rarely or never issued within smaller bins, for example, for a small sample of probability forecasts from a large ensemble system (Atger 2004). Third, it can allow cleaner comparison of Brier score components for forecasting systems having different numbers of ensemble forecasts (Mullen and Buizza 2002;Ferro 2007).…”

Section: Introductionmentioning

confidence: 99%

Two Extra Components in the Brier Score Decomposition

Stephenson

Coelho

Jolliffe

2008

Weather and Forecasting

View full text Add to dashboard Cite

The Brier score is widely used for the verification of probability forecasts. It also forms the basis of other frequently used probability scores such as the rank probability score. By conditioning (stratifying) on the issued forecast probabilities, the Brier score can be decomposed into the sum of three components: uncertainty, reliability, and resolution. This Brier score decomposition can provide useful information to the forecast provider about how the forecasts can be improved.Rather than stratify on all values of issued probability, it is common practice to calculate the Brier score components by first partitioning the issued probabilities into a small set of bins. This note shows that for such a procedure, an additional two within-bin components are needed in addition to the three traditional components of the Brier score. The two new components can be combined with the resolution component to make a generalized resolution component that is less sensitive to choice of bin width than is the traditional resolution component. The difference between the generalized resolution term and the conventional resolution term also quantifies how forecast skill is degraded when issuing categorized probabilities to users. The ideas are illustrated using an example of multimodel ensemble seasonal forecasts of equatorial sea surface temperatures.

show abstract

“…The analog and MOS methods require longterm training data of at least several years of historical forecasts. Limitations to the value of training samples arise from insufficient numbers (Atger 2003) and interdependence of the samples (Eckel and Walters 1998).…”

Section: Introductionmentioning

confidence: 99%

Short-Range Precipitation Forecasts from Time-Lagged Multimodel Ensembles during the HMT-West-2006 Campaign

Yuan

McGinley

Schultz

et al. 2008

Journal of Hydrometeorology

View full text Add to dashboard Cite

High-resolution (3 km) time-lagged (initialized every 3 h) multimodel ensembles were produced in support of the Hydrometeorological Testbed (HMT)-West-2006 campaign in northern California, covering the American River basin (ARB). Multiple mesoscale models were used, including the Weather Research and Forecasting (WRF) model, Regional Atmospheric Modeling System (RAMS), and fifth-generation Pennsylvania State University-National Center for Atmospheric Research Mesoscale Model (MM5). Shortrange (6 h) quantitative precipitation forecasts (QPFs) and probabilistic QPFs (PQPFs) were compared to the 4-km NCEP stage IV precipitation analyses for archived intensive operation periods (IOPs). The two sets of ensemble runs (operational and rerun forecasts) were examined to evaluate the quality of highresolution QPFs produced by time-lagged multimodel ensembles and to investigate the impacts of ensemble configurations on forecast skill. Uncertainties in precipitation forecasts were associated with different models, model physics, and initial and boundary conditions. The diabatic initialization by the Local Analysis and Prediction System (LAPS) helped precipitation forecasts, while the selection of microphysics was critical in ensemble design. Probability biases in the ensemble products were addressed by calibrating PQPFs. Using artificial neural network (ANN) and linear regression (LR) methods, the bias correction of PQPFs and a cross-validation procedure were applied to three operational IOPs and four rerun IOPs. Both the ANN and LR methods effectively improved PQPFs, especially for lower thresholds. The LR method outperformed the ANN method in bias correction, in particular for a smaller training data size. More training data (e.g., one-season forecasts) are desirable to test the robustness of both calibration methods.

show abstract

Spatial and Interannual Variability of the Reliability of Ensemble-Based Probabilistic Forecasts: Consequences for Calibration

Cited by 81 publications

References 19 publications

Ensemble forecasting

Ensemble forecasting

Two Extra Components in the Brier Score Decomposition

Short-Range Precipitation Forecasts from Time-Lagged Multimodel Ensembles during the HMT-West-2006 Campaign

Contact Info

Product

Resources

About