Comparative Verification of Recent Quantitative Precipitation Forecasts in the National Weather Service: A Simple Approach for Scoring Forecast Accuracy

Charba, Jerome P.; Reynolds, David W.; McDonald, Brett E.; Carter, Gary M.

doi:10.1175/1520-0434(2003)018<0161:cvorqp>2.0.co;2

Cited by 32 publications

(24 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The observed data quality control procedures have been enhanced through improved displays of range checking and spatial anomaly information as well as improved displays of the data itself. The precipitation forecast process has been updated because it was found that the QPF had a patchwork characteristic when the forecasts from multiple WFOs were aggregated into a single composite (National Weather Service 1999; Charba et al 2003). The QPF is now generated at the RFC rather than WFOs.…”

Section: Daily Forecast Process the Daily Forecast Process Beginsmentioning

confidence: 99%

Hydrologic Verification: A Call for Action and Collaboration

Welles¹,

Sorooshian²,

Carter³

et al. 2007

Bulletin of the American Meteorological Society

View full text Add to dashboard Cite

A brief verification study of river forecasts suggests the need to link river forecast process improvements more closely to forecast verification results. V erification must be an integral element of forecasting. Well-structured verification provides a means to improve forecast skill, to communicate with nonforecasters regarding resource needs, and to help forecast users optimize their decision making. Within the hydrology community however, few have focused any attention on verifying river forecasts. As a step toward encouraging hydrologists to verify their forecasts, this paper presents a verification study of National Oceanic and Atmospheric Administration/ National Weather Service (NWS) deterministic riverstage forecasts at 15 locations. The results of this study AMERICAN METEOROLOGICAL SOCIETY suggest that the hydrologic research and operations communities must join together to review, evaluate, and reconstruct the methods by which they update the hydrologic forecast process.The verification results described in this paper are for river-stage forecasts issued by NWS River Forecast Centers (RFCs). The NWS RFCs sit at the center of the U.S. flood warning capability. They provide guidance to the NWS Weather Forecast Offices (WFOs), which in turn issue flood watches and warnings. The NWS RFCs coordinate with other state and federal water management agencies when they issue their forecasts to ensure dam operations, irrigation demand and the like are integrated into the forecasts. There are 13 RFCs across the country, and each one is responsible for a different set of basins. A more detailed description of NWS river-forecasting operations can be found in Stallings and Wenzel (1995), Larson et al. (1995), andFread et al. (1995). In addition, the NWS RFCs describe their operations on their home pages, which can be found via the NWS home page (online at http://nws.noaa.gov).

show abstract

Section: Daily Forecast Process the Daily Forecast Process Beginsmentioning

confidence: 99%

Hydrologic Verification: A Call for Action and Collaboration

Welles¹,

Sorooshian²,

Carter³

et al. 2007

Bulletin of the American Meteorological Society

View full text Add to dashboard Cite

show abstract

“…-it is not obvious that the forecaster has an added value: some post processing method is competitive or superior to human forecast (see for instance Baars et al, 2005;Charba et al, 2003;Sanders et al, 1986;Roebber et al, 1996c).…”

Section: Turco and M Milelli: Added Value In Qpfmentioning

confidence: 99%

The forecaster's added value in QPF

Turco

Milelli

2010

Adv. Geosci.

View full text Add to dashboard Cite

Abstract.To the authors' knowledge there are relatively few studies that try to answer this question: "Are humans able to add value to computer-generated forecasts and warnings?". Moreover, the answers are not always positive. In particular some postprocessing method is competitive or superior to human forecast. Within the alert system of ARPA Piemonte it is possible to study in an objective manner if the human forecaster is able to add value with respect to computergenerated forecasts. Every day the meteorology group of the Centro Funzionale of Regione Piemonte produces the HQPF (Human Quantitative Precipitation Forecast) in terms of an areal average and maximum value for each of the 13 warning areas, which have been created according to meteohydrological criteria. This allows the decision makers to produce an evaluation of the expected effects by comparing these HQPFs with predefined rainfall thresholds. Another important ingredient in this study is the very dense non-GTS (Global Telecommunication System) network of rain gauges available that makes possible a high resolution verification. In this work we compare the performances of the latest three years of QPF derived from the meteorological models COSMO-I7 (the Italian version of the COSMO Model, a mesoscale model developed in the framework of the COSMO Consortium) and IFS (the ECMWF global model) with the HQPF. In this analysis it is possible to introduce the hypothesis test developed by Hamill (1999), in which a confidence interval is calculated with the bootstrap method in order to establish the real difference between the skill scores of two competitive forecasts. It is important to underline that the conclusions refer to the analysis of the Piemonte operational alert system, so they cannot be directly taken as universally true. But we think that some of the main lessons that can be derived from this study could be useful for the meteCorrespondence to: M. Turco (mturco@am.ub.es) orological community. In details, the main conclusions are the following:-despite the overall improvement in global scale and the fact that the resolution of the limited area models has increased considerably over recent years, the QPF produced by the meteorological models involved in this study has not improved enough to allow its direct use: the subjective HQPF continues to offer the best performance for the period +24 h/+48 h (i.e. the warning period in the Piemonte system);-in the forecast process, the step where humans have the largest added value with respect to mathematical models, is the communication. In fact the human characterization and communication of the forecast uncertainty to end users cannot be replaced by any computer code; -eventually, although there is no novelty in this study, we would like to show that the correct application of appropriated statistical techniques permits a better definition and quantification of the errors and, mostly important, allows a correct (unbiased) communication between forecasters and decision makers.

show abstract

“…After decades of development, short-range and, especially, medium-range precipitation forecasts, have been greatly improved by employing ensemble prediction systems (Charba and Klein, 1980;Charba et al, 2003). However, single EPS usually have restrictions in capturing specific atmospheric conditions; consequently multi-model prediction system (MMS) and probabilistic prediction were developed by considering the characteristics of many EPS.…”

Section: Introductionmentioning

confidence: 99%

“…Buizza (2008) summarized two of the main advantages of ensemble-based probabilistic forecasts as: the ability of an EPS to predict the most likely scenario; and the ability of an EPS to predict the probability of occurrence of any event, and provide more consistent successive forecasts. Many other meteorologists (Wandishin and Mullen, 2001;Verbunt 2007;Pappenberger 2008;Fowler et al, 2007) have documented the advantages Published by Copernicus Publications on behalf of the European Geosciences Union.…”

Section: Introductionmentioning

confidence: 99%

Verification and comparison of probabilistic precipitation forecasts using the TIGGE data in the upriver of Huaihe Basin

Liang

Tian

et al. 2011

Adv. Geosci.

View full text Add to dashboard Cite

Abstract. The precipitation forecasts of three ensemble prediction systems (EPS) and two multi-model ensemble prediction systems (MM EPS) were assessed by comparing with observations from 19 rain gauge stations located in the Dapoling-Wangjiaba sub-catchment of Huaihe Basin for the period from 1 July to 6 August 2008. The sample Probabilistic Distribution Functions (PDF) of gamma distribution, the Relative Operating Characteristic (ROC) diagrams, the percentile precipitation and a heavy rainfall event are analyzed to evaluate the performances of the single and multi-model ensemble prediction system (EPS).The three EPS were from the China Meteorological Administration (CMA); the United States National Centre for Environment Predictions (NCEP); and the European Centre for Medium-Range Weather Forecasts (ECMWF), all were obtained from the TIGGE-CMA archiving centre (THOR-PEX Interactive Grand Global Ensemble, TIGGE). The MM EPS were created using the equal weighting method for every ensemble member over the test area, the first ( MM-1) consisted of all three EPS, the second (MM-2) consisted of the ECMWF and NCEP EPS.The results demonstrate the level of correspondence between deterioration in predictive skill and extended lead time. Compared with observations and with a lead time of one day, ECMWF performs a little better than other centre's. With over five days in advance, all the three EPS and the two MM EPS don't give reliable probabilistic precipitation forecasts. Both MM EPS can outperform CMA and NCEP for most of the forecasted days, but still perform a little worse than ECMWF. Though variation of daily percentile precipitaCorrespondence to: F.-Y. Tian (tianfy@cma.gov.cn) tion and ROC areas show MM-2 outperforms MM-1, gamma distribution indicates much similar performances for all 10-day forecast, and neither is superior to ECMWF.

show abstract

Comparative Verification of Recent Quantitative Precipitation Forecasts in the National Weather Service: A Simple Approach for Scoring Forecast Accuracy

Cited by 32 publications

References 27 publications

Hydrologic Verification: A Call for Action and Collaboration

Hydrologic Verification: A Call for Action and Collaboration

The forecaster's added value in QPF

Verification and comparison of probabilistic precipitation forecasts using the TIGGE data in the upriver of Huaihe Basin

Contact Info

Product

Resources

About