ABSTRACT:Research and development of new verification strategies and reassessment of traditional forecast verification methods has received a great deal of attention from the scientific community in the last decade. This scientific effort has arisen from the need to respond to changes encompassing several aspects of the verification process, such as the evolution of forecasting systems, or the desire for more meaningful verification approaches that address specific forecast user requirements. Verification techniques that account for the spatial structure and the presence of features in forecast fields, and which are designed specifically for high-resolution forecasts have been developed. The advent of ensemble forecasts has motivated the re-evaluation of some of the traditional scores and the development of new verification methods for probability forecasts. The expected climatological increase of extreme events and their potential socio-economical impacts have revitalized research studies addressing the challenges concerning extreme event verification. Verification issues encountered in the operational forecasting environment have been widely discussed, verification needs for different user communities have been identified, and models to assess the forecast value for specific users have been proposed. Proper verification practice and correct interpretation of verification statistics has been extensively promoted with recent publications and books, tutorials and workshops, and the development of open-source software and verification tools. This paper addresses some of the current issues in forecast verification, reviews some of the most recently developed verification techniques, and provides recommendations for future research.

No abstract

The demand for verification of forecasting systems to ascertain their strengths and weaknesses is increasing dramatically as models evolve more rapidly. Precipitation forecasts have always been of great interest to forecasters as they influence daily life. The recent flooding over Europe has also shown how important it is to know how models can reproduce these events. In this paper we address the issue of precipitation verification, starting from the assumption that model spatial scales have to be verified against data representing similar scales. Only in this way we are able to determine the skill of our forecasting system. A high-resolution observing network over the Alpine region has been used to reconstruct the observed precipitation field. It contains smoothed small-scale variability and represents with sufficient accuracy the average behaviour of the observed field in the model grid box. It is shown that verification against irregular and scattered observations are highly influenced by the variability of the precipitation in a grid-box. A precipitation analysis is, therefore, important if model skills have to be defined.

During the 2006 African Monsoon Multidisciplinary Analysis (AMMA) field experiment, an unprecedented number of soundings were performed in West Africa. However, due to technical problems many of these soundings did not reach the Global Telecommunication System and therefore they could not be included in the operational numerical weather prediction (NWP) analyses. This issue, together with the realization that there was a significant bias in the radiosonde humidity, led to the conclusion that a re-analysis effort was necessary. This re-analysis was performed at the European Centre for Medium-Range Weather Forecasts (ECMWF) spanning the wet monsoon season of 2006 from May-September. The key features of the ECMWF AMMA re-analysis are presented, including the use of a newer model version with improved physics, all the AMMA radiosonde data available from the AMMA database and a new radiosonde humidity bias-correction scheme. Dataimpact experiments show that there is a benefit from these observations, but also highlight large model physics biases over the Sahel that cause a short-lived impact of the observations on the model forecast. The AMMA re-analysis is compared with independent observations to investigate the biases in the different parts of the physics. In the framework of the AMMA project, a hybrid dataset was developed to provide a best estimate of the different terms of the water cycle. This hybrid dataset is used to evaluate the improvement achieved from the use of extra AMMA observations and of a radiosonde humidity bias-correction scheme in the water cycle of the West African monsoon. Finally, future model developments that offer promising improvements in the water cycle are discussed.

A methodology for evaluating ensemble forecasts, taking into account observational uncertainties for catchment-based precipitation averages, is introduced. Probability distributions for mean catchment precipitation are derived with the Generalized Likelihood Uncertainty Estimation (GLUE) method. The observation uncertainty includes errors in the measurements, uncertainty as a result of the inhomogeneities in the rain gauge network, and representativeness errors introduced by the interpolation methods. The closeness of the forecast probability distribution to the observed fields is measured using the Brier skill score, rank histograms, relative entropy, and the ratio between the ensemble spread and the error of the ensemblemedian forecast (spread-error ratio). Four different methods have been used to interpolate observations on the catchment regions. Results from a 43-day period (20 July-31 August 2002) show little sensitivity to the interpolation method used. The rank histograms and the relative entropy better show the effect of introducing observation uncertainty, although this effect on the Brier skill score and the spread-error ratio is not very large. The case study indicates that overall observation uncertainty should be taken into account when evaluating forecast skill.