Golden Rule of Forecasting: Be Conservative

Armstrong, J. Scott; Green, Kesten C.; Graefe, Andreas

doi:10.2139/ssrn.2643546

Cited by 23 publications

(28 citation statements)

References 114 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our three-factor framework is broadly consistent with the conservative tendency observed in western politics 8 and business management, 9 and concurs with the conservatism of many climate scientists. 10 One overarching assumption behind our framework is that experts are rational actors in their decision-making and judgment.…”

Section: Experts' Risk Perception and Rational Choicesupporting

confidence: 85%

Experts’ conservative judgment and containment of COVID-19 in early outbreak

Liu

et al. 2020

Journal of Chinese Governance

View full text Add to dashboard Cite

We present the case of the outbreak and containment of COVID-19 in its early stage and analyze the causes for conservative judgment of the disease control experts who are blamed for delayed action and inadequate response by the government. We find that the disease control experts have a clear tendency and preference to make conservative judgment about the epidemic risk of a low probability and high impact new disease. This conservative preference may have been a major factor in the judging and communicating the risk of COVID-19 epidemic. The experts' perception of the epidemic risk is affected by their habitual thinking facing a low probability event with uncertain progression. Their conservative preference is augmented by political concerns due to the high social and political impact of the event and the potential political consequence of a false judgment. Balancing personal payoffs of decision options, the experts' rational choice would lean towards a conservative decision. We propose a three-factor decision model that integrates habitual thinking, political concerns and rational choice to explain the experts' conservative judgment, and corroborate the model with evidences from the case. We conclude the paper with policy implications for improving experts' role in public health emergency response. ARTICLE HISTORY

show abstract

Section: Experts' Risk Perception and Rational Choicesupporting

confidence: 85%

Experts’ conservative judgment and containment of COVID-19 in early outbreak

Liu

et al. 2020

Journal of Chinese Governance

View full text Add to dashboard Cite

show abstract

“…In sum, then, the structured procedure that we developed for predicting ad effectiveness from evidence-based persuasion principles is based on evidence-based forecasting principles. In particular, the procedure follows three forecasting guidelines described by Armstrong, Green, and Graefe (2015): (1) use prior information to select variables and determine effect sizes, (2) use all available information, and (3) combine judgments.…”

Section: Creating a Persuasion Principles Indexmentioning

confidence: 99%

Predictive Validity of Evidence-Based Persuasion Principles: An Application of the Index Method

Armstrong

Green

et al. 2015

SSRN Journal

Self Cite

View full text Add to dashboard Cite

Purpose: To test whether a structured application of persuasion principles might help improve advertising decisions. Evidence-based principles are currently used to improve decisions in other complex situations, such as those faced in engineering and medicine.Approach: Scores were calculated from the ratings of 17 self-trained novices who rated 96 matched pairs of print advertisements for adherence to evidence-based persuasion principles. Predictions from traditional methods-10,809 unaided judgments from novices and 2,764 judgments from people with some expertise in advertising, and 288 copy-testing predictions from 5,285 judgments-provided benchmarks.Findings: The higher adherence to principles' consensus score correctly predicted the more effective ad for 75% of the pairs. Copy testing was correct for 59% and expert judgment was correct for 55%. Guessing would provide 50% accurate predictions. Combining judgmental predictions led to substantial improvements in accuracy.Research limitations: Ads for high-involvement utilitarian products were tested on the assumption that persuasion principles would be more effective for such products. The measure of effectiveness that was available-day-after-recall-is a proxy for persuasion or behavioral measures.Practical implications: Pretesting ads by assessing adherence to evidence-based persuasion principles in a structured way helps in deciding which ads would be best to run. Such a procedure also identifies how to make an ad more effective.Originality: This is the first study in marketing, and in advertising specifically, to test the predictive validity of evidence-based principles. In addition, the study provides the first test of the predictive validity of the index method for a marketing problem. AbstractPurpose: To test whether a structured application of persuasion principles might help improve advertising decisions. Evidence-based principles are currently used to improve decisions in other complex situations, such as those faced in engineering and medicine.Approach: Scores were calculated from the ratings of 17 self-trained novices who rated 96 matched pairs of print advertisements for adherence to evidence-based persuasion principles. Predictions from traditional methods-10,809 unaided judgments from novices and 2,764 judgments from people with some expertise in advertising, and 288 copy-testing predictions from 5,285 judgments-provided benchmarks. Findings:The higher adherence to principles' consensus score correctly predicted the more effective ad for 75% of the pairs. Copy testing was correct for 59%, and expert judgment was correct for 55%. Guessing would provide 50% accurate predictions.Combining of judgmental predictions led to substantial improvements in accuracy.Research limitations: Ads for high-involvement utilitarian products were tested on the assumption that persuasion principles would be more effective for such products. The measure of effectiveness that was available-day-after-recall-is a proxy for persuasion or behavioral measures. Practical implica...

show abstract

“…Unlike standard deviation that results from poor forecasting, bias is often managerially introduced through forecast overrides attempting to address asymmetric cost structures, compensate for anticipated events, and generally hedge against uncertainty. Therefore, bias may be managed (Armstrong et al, 2014;Fildes et al, 2009). Our findings suggest that introducing such bias should be done with great caution, following a cost analysis, and for forecasts that typically do not have large errors.…”

Section: Introductionmentioning

confidence: 86%

“…As such, our finding strongly suggests that introducing such bias should be done with caution, particularly for forecasts with large errors. Companies should consider the well documented guidelines for managing bias (Fildes & Goodwin, 2007;Armstrong et al, 2014). Mechanisms that automatically track bias in addition to more traditional forecast error measures may be a good investment if currently not in place.…”

Section: Conclusion and Future Researchmentioning

confidence: 99%

See 1 more Smart Citation

Impact of Bias Magnification on Supply Chain Costs: The Mitigating Role of Forecast Sharing

Sanders

Graman

2016

Decision Sciences

View full text Add to dashboard Cite

The impact of forecast error magnification on supply chain cost has been well documented. Unlike past studies that measure forecast error in terms of forecast standard deviation, our study extends research to consider the impact of forecast bias, and the complex interaction between these variables. Simulating a two-stage supply chain using realistic cost data we test the impact of bias magnification comparing two scenarios: one with forecast sharing between retailer and supplier, and one without. We then corroborate findings via survey data. Results show magnification of forecast bias to have a considerably greater impact on supply chain cost than magnification of forecast standard deviation. Particularly damaging is high bias in the presence of high forecast standard deviation. Forecast sharing is found to mitigate the impact of forecast error, however, primarily at higher levels of forecast standard deviation. At low levels of forecast standard deviation the benefits are not significant suggesting that engaging in such mitigation strategies may be less effective when there is little opportunity for improvement in accuracy. Furthermore, forecast sharing is found to be much less effective against high levels of bias. This is an important finding as managers often deliberately bias their forecasts and underscores the importance of exercising caution even with forecast sharing, particularly for forecasts that have inherently large errors. The findings provide a deeper understanding of the impact of forecast errors, suggest limitations of forecast sharing, and offer implications for research and practice alike.

show abstract

Golden Rule of Forecasting: Be Conservative

Cited by 23 publications

References 114 publications

Experts’ conservative judgment and containment of COVID-19 in early outbreak

Experts’ conservative judgment and containment of COVID-19 in early outbreak

Predictive Validity of Evidence-Based Persuasion Principles: An Application of the Index Method

Impact of Bias Magnification on Supply Chain Costs: The Mitigating Role of Forecast Sharing

Contact Info

Product

Resources

About