The receiver operating characteristic (ROC) curve is a two-dimensional measure of classification performance. The area under the ROC curve (AUC) is a scalar measure gauging one facet of performance. In this short article, five idealized models are utilized to relate the shape of the ROC curve, and the area under it, to features of the underlying distribution of forecasts. This allows for an interpretation of the former in terms of the latter. The analysis is pedagogical in that many of the findings are already known in more general (and more realistic) settings; however, the simplicity of the models considered here allows for a clear exposition of the relation. For example, although in general there are many reasons for an asymmetric ROC curve, the models considered here clearly illustrate that an asymmetry in the ROC curve can be attributed to unequal widths of the distributions. Furthermore, it is shown that AUC discriminates well between “good” and “bad” models, but not between good models.

No abstract

The rank histogram (RH) is a visual tool for assessing the reliability of ensemble forecasts (i.e., the degree to which the forecasts and the observations have the same distribution). But it is already known that in certain situations it conveys misleading information. Here, it is shown that a temporal correlation can lead to a misleading RH, but such a correlation contributes only to the sampling variability of the RH, and so it is accounted for by producing a RH that explicitly displays sampling variability. A simulation is employed to show that the variance within each ensemble member (i.e., climatological variance), the correlation between ensemble members, and the correlation between the observations and the forecasts, all have a confounding effect on the RH, making it difficult to use the RH for assessing the climatological component of forecast reliability. It is proposed that a ''residual'' quantile-quantile plot (denoted R-Q-Q plot) is better suited than the RH for assessing the climatological component of forecast reliability. Then, the RH and R-Q-Q plots for temperature and wind speed forecasts at 90 stations across the continental United States are computed. A wide range of forecast reliability is noted. For some stations, the nonreliability of the forecasts can be attributed to bias and/or under-or overclimatological dispersion. For others, the difference between the distributions can be traced to lighter or heavier tails in the distributions, while for other stations the distributions of the forecasts and the observations appear to be completely different. A spatial signature is also noted and discussed briefly.

The divergent part of the one-loop off-shell effective action is computed for a single scalar field coupled to the Ricci curvature of 2D gravity (cφR), and self interacting by an arbitrary potential term V (φ). The Vilkovisky-DeWitt effective action is used to compute gauge-fixing independent results. In our background field/covariant gauge we find that the Liouville theory is finite on shell. Off-shell, we find a large class of renormalizable potentials which include the Liouville potential. We also find that for backgrounds satisfying R = 0, the Liouville theory is finite off shell, as well.

A set of 14 scalar, nonprobabilistic measures-some old, some new-is examined in the rare-event situation. The set includes measures of accuracy, association, discrimination, bias, and skill. It is found that all measures considered herein are inequitable in that they induce under-or overforecasting. One condition under which such bias is not induced (for some of the measures) is when the underlying class-conditional distributions are Gaussian (normal) and equivariant.

A statistical method referred to as cluster analysis is employed to identify features in forecast and observation fields. These features qualify as natural candidates for events or objects in terms of which verification can be performed. The methodology is introduced and illustrated on synthetic and real quantitative precipitation data. First, it is shown that the method correctly identifies clusters that are in agreement with what most experts might interpret as features or objects in the field. Then, it is shown that the verification of the forecasts can be performed within an event-based framework, with the events identified as the clusters. The number of clusters in a field is interpreted as a measure of scale, and the final “product” of the methodology is an “error surface” representing the error in the forecasts as a function of the number of clusters in the forecast and observation fields. This allows for the examination of forecast error as a function of scale.

No abstract

The correlation between tornadic activity in several regions of the United States and the monthly mean sea surface temperature over four zones in the tropical Pacific Ocean is examined. Tornadic activity is gauged with two mostly independent measures: the number of tornadoes per month, and the number of tornadic days per month. Within the assumptions set forth for the analysis, it is found that there appears to exist a statistically significant but very weak correlation between sea surface temperature in the Pacific Ocean and tornadic activity in the United States, with the strength and significance of the correlation depending on the coordinates at which the sea surface temperatures are assessed and the geographic region of the United States. The strongest evidence found is for the correlation between the number of days with strong and violent (F2 and greater) tornadoes in an area that runs from Illinois to the Atlantic Coast, and Kentucky to Canada and a cool sea surface temperature in the central tropical Pacific. However, there is only about a 53% chance of this relationship occurring in a specific month.