On Subjective Probability Forecasting

Sanders, Frederick

doi:10.1175/1520-0450(1963)002<0191:ospf>2.0.co;2

Cited by 192 publications

(125 citation statements)

References 0 publications

Supporting

Mentioning

115

Contrasting

Unclassified

Order By: Relevance

“…The ideal forecast is the sharpest of all auto-calibrated forecasts, because it is characterized by the maximal expected score. This fact is seen from the classical partitioning of the Brier score for dichotomous outcomes into the sum of sharpness and calibration ("validity") terms developed in Sanders (1963). For an auto-calibrated forecast the calibration term is zero.…”

Section: Testing Vs Scoring Rulesmentioning

confidence: 99%

Evaluating Density Forecasts: A Comment

Tsyplakov

2011

SSRN Journal

View full text Add to dashboard Cite

Section: Testing Vs Scoring Rulesmentioning

confidence: 99%

Evaluating Density Forecasts: A Comment

Tsyplakov

2011

SSRN Journal

View full text Add to dashboard Cite

“…; K, where K is the total number of categories. Then the calibration index (CI) is given by (Murphy, 1973;Sanders, 1963):…”

Section: Murphy Decompositionmentioning

confidence: 99%

Professional vs. amateur judgment accuracy: The case of foreign exchange rates

Önkal

Yates

Şımga-Muğan

et al. 2003

Organizational Behavior and Human Decision Processes

View full text Add to dashboard Cite

Highly knowledgeable people often fail to achieve highly accurate judgments, a phenomenon sometimes called the ''processperformance paradox.'' The present research tested for this paradox in foreign exchange (FX) rate forecasting. Forty professional and 57 sophisticated amateur forecasters made one-day and one-week-ahead FX predictions in deterministic and probabilistic formats. Among the conclusions indicated by the results are: (a) professional accuracy usually surpasses amateur accuracy, although many amateurs outperform many professionals; (b) professionals appear to achieve high proficiency via heavy reliance on predictive information (unlike what has been observed before, e.g., for stock prices); (c) forecast format strongly affects judgment accuracy and processes; and (d) apparent overconfidence can transform itself into underconfidence depending on when and how forecasters must articulate their confidence.

show abstract

“…This makes it reasonable to use MSE or LogLoss as measures to evaluate calibration methods, since the 'separability components' are not affected. This is clearly seen through the so-called "decompositions of the Brier score" (Sanders, 1963;Murphy, 1972) included in a previous section in this chapter.…”

Section: Calibration Methods For Type Cpmentioning

confidence: 78%

Calibration of Machine Learning Models

Bella

Ferri

Hernández-Orallo

et al. 2010

Handbook of Research on Machine Learning Applications and Trends

View full text Add to dashboard Cite

Evaluation of machine learning methods is a crucial step before application, because it is essential to assess how good a model will behave for every single case. In many real applications, not only the "total" or the "average" of the error of the model is important but it is also important to know how this error is distributed or how well confidence or probability estimations are made. However, many machine learning techniques are good in overall results but have a bad distribution /assessment of the error.In these cases, calibration techniques have been developed as postprocessing techniques which aim at improving the probability estimation or the error distribution of an existing model.In this chapter, we present the most usual calibration techniques and calibration measures. We cover both classification and regression, and we establish a taxonomy of calibration techniques, while then paying special attention to probabilistic classifier calibration.

show abstract

On Subjective Probability Forecasting

Cited by 192 publications

References 0 publications

Evaluating Density Forecasts: A Comment

Evaluating Density Forecasts: A Comment

Professional vs. amateur judgment accuracy: The case of foreign exchange rates

Calibration of Machine Learning Models

Contact Info

Product

Resources

About