Skill in Precipitation Forecasting in the National Weather Service

Charba, Jerome P.; Klein, William H.

doi:10.1175/1520-0477(1980)061<1546:sipfit>2.0.co;2

Cited by 36 publications

(28 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This would expand on the work carried out at the National Meteorological Center since 1961 (Charba and Klein, 1980 …”

Section: Discussionmentioning

confidence: 99%

Have Precipitation Forecasts Improved?

Ramage

1982

Bulletin of the American Meteorological Society

View full text Add to dashboard Cite

Twelve years of precipitation forecasts made at National Weather Service forecast offices have been evaluated with a view to determining accuracy trends. Relative precipitation frequency, which proved to be negatively correlated to forecast accuracy, accounted for generally more than 80% of the variance. Winter forecasts were better than summer forecasts, and accuracy differed insignificantly among the eastern, southern, central, and western regions. After the contribution of relative precipitation frequency was removed, accuracy improvement, amounting to about 1% in seven years, could be detected only in the eastern and central region winter forecasts.

show abstract

“…This would expand on the work carried out at the National Meteorological Center since 1961 (Charba and Klein, 1980 …”

Section: Discussionmentioning

confidence: 99%

Have Precipitation Forecasts Improved?

Ramage

1982

Bulletin of the American Meteorological Society

View full text Add to dashboard Cite

show abstract

“…Judges' calibration can be empirically evaluated by observing their probability assessments, verifying the associated propositions, and then observing the proportion that is true in each response category." With a few exceptions, such as weather forecasters who make daily precipitation forecasts aided by computer models and receive regular feedback on how well they are performing (42,43), most people making subjective judgments are not very well calibrated. Fig.…”

Section: Ubiquitous Overconfidencementioning

confidence: 99%

“…Fig. 5 shows examples of very poorly calibrated results from clinical diagnosis of pneumonia (44) to very well-calibrated probabilistic precipitation judgments by US weather forecasters (43).…”

Section: Ubiquitous Overconfidencementioning

confidence: 99%

Use (and abuse) of expert elicitation in support of decision making for public policy

Morgan

2014

Proc. Natl. Acad. Sci. U.S.A.

524

467

View full text Add to dashboard Cite

The elicitation of scientific and technical judgments from experts, in the form of subjective probability distributions, can be a valuable addition to other forms of evidence in support of public policy decision making. This paper explores when it is sensible to perform such elicitation and how that can best be done. A number of key issues are discussed, including topics on which there are, and are not, experts who have knowledge that provides a basis for making informed predictive judgments; the inadequacy of only using qualitative uncertainty language; the role of cognitive heuristics and of overconfidence; the choice of experts; the development, refinement, and iterative testing of elicitation protocols that are designed to help experts to consider systematically all relevant knowledge when they make their judgments; the treatment of uncertainty about model functional form; diversity of expert opinion; and when it does or does not make sense to combine judgments from different experts. Although it may be tempting to view expert elicitation as a low-cost, low-effort alternative to conducting serious research and analysis, it is neither. Rather, expert elicitation should build on and use the best available research and analysis and be undertaken only when, given those, the state of knowledge will remain insufficient to support timely informed assessment and decision making.

show abstract

“…In addition, some experts, such as meteorologists (20,21) and bridge players (22), are very well calibrated. Intelligence analysts share some similarities with these experts.…”

mentioning

confidence: 99%

Accuracy of forecasts in strategic intelligence

Mandel

Barnes

2014

Proc. Natl. Acad. Sci. U.S.A.

114

View full text Add to dashboard Cite

The accuracy of 1,514 strategic intelligence forecasts abstracted from intelligence reports was assessed. The results show that both discrimination and calibration of forecasts was very good. Discrimination was better for senior (versus junior) analysts and for easier (versus harder) forecasts. Miscalibration was mainly due to underconfidence such that analysts assigned more uncertainty than needed given their high level of discrimination. Underconfidence was more pronounced for harder (versus easier) forecasts and for forecasts deemed more (versus less) important for policy decision making. Despite the observed underconfidence, there was a paucity of forecasts in the least informative 0.4-0.6 probability range. Recalibrating the forecasts substantially reduced underconfidence. The findings offer cause for tempered optimism about the accuracy of strategic intelligence forecasts and indicate that intelligence producers aim to promote informativeness while avoiding overstatement.forecasting | prediction | intelligence analysis | quality control | recalibration

show abstract

Skill in Precipitation Forecasting in the National Weather Service

Cited by 36 publications

References 18 publications

Have Precipitation Forecasts Improved?

Have Precipitation Forecasts Improved?

Use (and abuse) of expert elicitation in support of decision making for public policy

Accuracy of forecasts in strategic intelligence

Contact Info

Product

Resources

About