PredictiveScores

Abstract

Predictive scores assess the quality of forecasts, by assigning a numerical score based on the forecast and on the event or value that materializes. Probabilistic forecasts take the form of a predictive probability distribution. Their evaluation calls for the use of proper scoring rules, such as the logarithmic score or the continuous ranked probability score. Effective point forecasting requires a directive in the form of a target functional, say a quantile or the mean of the predictive distribution. It is then critical for the scoring function to be decision theoretically consistent, so that the expected score is minimized when following the directive. For example, the squared error is consistent when the point forecast is desired to be a mean, and the absolute error is consistent when we seek a median.