The Finley Affair: A Signal Event in the History of Forecast Verification

Abstract

In 1884 a paper by J. P. Finley appeared in the American Meteorological Journal describing the results of an experimental tornado forecasting program in the central and eastern United States. Finley's paper reported ''percentages of verifications'' exceeding 95%, where this index of performance was defined as the percentage of correct tornado/no-tornado forecasts. Within six months, three papers had appeared that identified deficiencies in Finley's method of verification and/or proposed alternative measures of forecasting performance in the context of this 2 1 2 verification problem. During the period from 1885 to 1893, several other authors in the United States and Europe, in most cases stimulated either by Finley's paper or by the three early responses, made noteworthy contributions to methods-oriented and practices-oriented discussions of issues related to forecast verification in general and verification of tornado forecasts in particular. The burst of verification-related activities during the period 1884-1893 is referred to here as the ''Finley affair.'' It marked the beginning of substantive conceptual and methodological developments and discussions in the important subdiscipline of forecast verification. This paper describes the events that constitute the Finley affair in some detail and attempts to place this affair in proper historical context from the perspective of the mid-1990s. Whatever their individual strengths and weaknesses, the measures introduced during the period from 1884 to 1893 have withstood important tests of time-for example, these measures have been rediscovered on one or more occasions and they are still widely used today (generally under names assigned since 1900). Moreover, many of the issues vis-à-vis forecast verification that were first raised during the Finley affair remain issues of considerable importance more than 100 years later.

Atmospheric Moisture and Cloud Cover Characteristics Forecast by AMPS*

“…Although the validation period is relatively short, it contains a diversity of weather features, suggesting that additional comparisons will yield similar results. We constructed 2 ϫ 2 contingency tables, along with their associated verification statistics [see Murphy (1996) and Wilks (2006) for a detailed description of these statistics], for each region in Fig. 8 as a means of quantitatively evaluating the pseudosatellite product, separated by low clouds (Table 2) and high clouds (Table 3).…”

“…Turning to the high cloud verification (Table 3), a greater level of skill is attained with the percent correct being above 80% in the majority of the regions, biases TABLE 2. The 2 ϫ 2 contingency tables and associated statistical skill scores as described in Murphy (1996) and Wilks (2006) for the low cloud performance of the pseudosatellite product for the regions defined in Fig. 8.…”

A new global real-time Lagrangian diagnostic system for stratosphere-troposphere exchange: evaluation during a balloon sonde campaign in eastern Canada

“…In general, the PC score artificially increases in cases of low event frequency (e.g. the Finley affair, Murphy, 1996).…”

Les progrès dans les prévisions : météorologie et économique

“…Le travail de pionnier de Finley (1884) a été étudié par nombre d'auteurs, y compris Murphy (1996), et la valeur d'un certain nombre de mesures pour prévi-sions binaires ont été calculées à partir du tableau de contingences 2 2 des avertissements de Finley. On peut donc attirer l'attention, pour des fins de comparaison avec des résultats récents, sur le fait que dans l'étude de Finley, l'indice de menace était de 0,228, la probabilité de détection de 0,549 et le taux de fausses alarmes de 0,720.…”