Predicting elections: Experts, polls, and fundamentals

Graefe, Andreas

doi:10.1017/s1930297500009219

Cited by 16 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ACE-IDEA data set (Hanea et al, 2021) includes forecasts on 155 events, but on average each forecaster only replied to about 19 queries. Other data sets consist of less queries to be predicted or answered (Graefe, 2018;Hanea et al, 2021;Karvetski et al, 2013;Prelec et al, 2017), of which the highest number of queries is about 80 (Prelec et al, 2017). However, 80 answers per forecaster is still a small number for modeling the forecasters' behavior, particularly if we divide the data into training and test sets and model the answers to true/false queries separately.…”

Section: Figure 1 Graphical Models Of One Of the Models Proposed Bymentioning

confidence: 99%

“…While statistical models are usually limited in applicability by requiring sufficiently large and complete data sets, human forecasts can overcome this limitation taking advantage of human experience and intuition (Clemen, 1989;Clemen & Winkler, 1986;Genest & Zidek, 1986). The probability estimates can be given either as forecasts of events, e.g., rain probabilities in meteorological science or probabilities for the outcomes of geopolitical events such as elections (Graefe, 2018;Turner et al, 2014), other binary classifications, or the quantification of the experts' confidence on a prediction or the answer to a specific question (Karvetski et al, 2013;Prelec et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A normative model for Bayesian combination of subjective probability estimates

Trick,

Rothkopf,

Jäkel

2023

Judgm. decis. mak.

View full text Add to dashboard Cite

Combining experts’ subjective probability estimates is a fundamental task with broad applicability in domains ranging from finance to public health. However, it is still an open question how to combine such estimates optimally. Since the beta distribution is a common choice for modeling uncertainty about probabilities, here we propose a family of normative Bayesian models for aggregating probability estimates based on beta distributions. We systematically derive and compare different variants, including hierarchical and non-hierarchical as well as asymmetric and symmetric beta fusion models. Using these models, we show how the beta calibration function naturally arises in this normative framework and how it is related to the widely used Linear-in-Log-Odds calibration function. For evaluation, we provide the new Knowledge Test Confidence data set consisting of subjective probability estimates of 85 forecasters on 180 queries. On this and another data set, we show that the hierarchical symmetric beta fusion model performs best of all beta fusion models and outperforms related Bayesian fusion models in terms of mean absolute error.

show abstract

Section: Figure 1 Graphical Models Of One Of the Models Proposed Bymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A normative model for Bayesian combination of subjective probability estimates

Trick,

Rothkopf,

Jäkel

2023

Judgm. decis. mak.

View full text Add to dashboard Cite

show abstract

“…Other research has previously explored the ways that automated approaches can benefit human forecasting in domains spanning judicial decisions (Jung et al, 2020), meteorology (Yu et al, 2011), health care (Goldstein et al, 2017), and defense (Scharre, 2016). Such methods range from simple averaging of human and model forecasts (Graefe, 2018), to expert-informed feature selection in regression models (Jung et al, 2020), to hybrid prediction markets (Nagar & Malone, 2012). The line between human and machine is often blurry: algorithms may incorporate human judgments as inputs (e.g., an interviewer’s ratings of job candidates), and human experts may decide the form and content of an algorithm.…”

mentioning

confidence: 99%

Automated update tools to augment the wisdom of crowds in geopolitical forecasting.

Summerville,

Widmer,

Minnery

et al. 2024

Decision

View full text Add to dashboard Cite

Despite the importance of predictive judgments, individual human forecasts are frequently less accurate than those of even simple prediction algorithms. At the same time, not all forecasts are amenable to algorithmic prediction. Here, we describe the evaluation of an automated prediction tool that enabled participants to create simple rules that monitored relevant indicators (e.g., commodity prices) to automatically update forecasts. We examined these rules in both a pool of previous participants in a geopolitical forecasting tournament (Study 1) and a naïve sample recruited from Mechanical Turk (Study 2). Across the two studies, we found that automated updates tended to improve forecast accuracy relative to initial forecasts and were comparable to manual updates. Additionally, making rules improved the accuracy of manual updates. Crowd forecasts likewise benefitted from rulebased updates. However, when presented with the choice of whether to accept, reject or adjust an automatic forecast update, participants showed little ability to discriminate between automated updates that were harmful versus beneficial to forecast accuracy. Simple prospective rule-based tools are thus able to improve forecast accuracy by offering accurate and efficient updates, but ensuring forecasters make use of tools remains a challenge.

show abstract

Aggregating predictions from experts: A review of statistical methods, experiments, and applications

McAndrew

Wattanachit

Gibson

et al. 2020

WIREs Computational Stats

View full text Add to dashboard Cite

Forecasts support decision making in a variety of applications. Statistical models can produce accurate forecasts given abundant training data, but when data is sparse or rapidly changing, statistical models may not be able to make accurate predictions. Expert judgmental forecasts—models that combine expert‐generated predictions into a single forecast—can make predictions when training data is limited by relying on human intuition. Researchers have proposed a wide array of algorithms to combine expert predictions into a single forecast, but there is no consensus on an optimal aggregation model. This review surveyed recent literature on aggregating expert‐elicited predictions. We gathered common terminology, aggregation methods, and forecasting performance metrics, and offer guidance to strengthen future work that is growing at an accelerated pace. This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical and Graphical Methods of Data Analysis > Multivariate Analysis

show abstract

Predicting elections: Experts, polls, and fundamentals

Cited by 16 publications

References 29 publications

A normative model for Bayesian combination of subjective probability estimates

A normative model for Bayesian combination of subjective probability estimates

Automated update tools to augment the wisdom of crowds in geopolitical forecasting.

Aggregating predictions from experts: A review of statistical methods, experiments, and applications

Contact Info

Product

Resources

About