Evaluating expert advice in forecasting: Users’ reactions to presumed vs. experienced credibility

Önkal, Dilek; Gönül, M. Sinan; Goodwin, Paul; Thomson, Mary E.; Öz, Esra

doi:10.1016/j.ijforecast.2015.12.009

Cited by 19 publications

(9 citation statements)

References 71 publications

(89 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, this behavior‐belief (in)congruity is another dimension of algorithmic advice taking that is under‐studied, and we believe that this may lead to false generalizations about the persuasive impact of algorithms. Past studies have found that statements or presumptions of trust in advisors do not necessarily correspond how influential their advice is in belief revision (Goodwin et al, 2013; Önkal et al, 2017). More specifically, higher levels of trust did not meaningfully increase persuasiveness compared to controls, though lower levels of trust did, albeit slightly.…”

Section: Introductionmentioning

confidence: 99%

Preference for human or algorithmic forecasting advice does not predict if and how it is used

Himmelstein

Budescu

2022

Behavioral Decision Making

View full text Add to dashboard Cite

Past research has found that people treat advice differently depending on its source.In many cases, people seem to prefer human advice to algorithms, but in others, there is a reversal, and people seem to prefer algorithmic advice. Across two studies, we examine the persuasiveness of, and judges' preferences for, advice from different sources when forecasting geopolitical events. We find that judges report domainspecific preferences, preferring human advice in the domain of politics and algorithmic advice in the domain of economics. In Study 2, participants report a preference for hybrid advice, that combines human and algorithmic sources, to either one on it's own regardless of domain. More importantly, we find that these preferences did not affect persuasiveness of advice from these different sources, regardless of domain.Judges were primarily sensitive to quantitative features pertaining to the similarity between their initial beliefs and the advice they were offered, such as the distance between them and the relative advisor confidence, when deciding whether to revise their initial beliefs in light of advice, rather than the source that generated the advice.

show abstract

Section: Introductionmentioning

confidence: 99%

Preference for human or algorithmic forecasting advice does not predict if and how it is used

Himmelstein

Budescu

2022

Behavioral Decision Making

View full text Add to dashboard Cite

show abstract

“…One intuitive research question that arises is how humans take the provided advice and use it to update their belief. On a human-to-human basis, Önkal et al [212] and Ayton & Önkal [213] investigated empirically the driving factors in using recommendations and advice. The authors assess whether experienced or presumed credibility has more impact on judges' readiness to use advice.…”

Section: Human Advice Taking and Belief Updatingmentioning

confidence: 99%

“…the credibility purveyed through the status of the advice giver, plays the biggest role in judgement adoption. Önkal et al [212] showed, experimentally, that advice from a forecaster with high experienced credibility received a higher weight, and a lower level of credibility did not affect the weighting negatively. High presumed credibility in turn did not result in an allocation of more weight to the model, although low presumed credibility resulted in a decrease.…”

Section: Human Advice Taking and Belief Updatingmentioning

confidence: 99%

A survey of human judgement and quantitative forecasting methods

et al. 2021

View full text Add to dashboard Cite

This paper's top-level goal is to provide an overview of research conducted in the many academic domains concerned with forecasting. By providing a summary encompassing these domains, this survey connects them, establishing a common ground for future discussions. To this end, we survey literature on human judgement and quantitative forecasting as well as hybrid methods that involve both humans and algorithmic approaches. The survey starts with key search terms that identified more than 280 publications in the fields of computer science, operations research, risk analysis, decision science, psychology and forecasting. Results show an almost 10-fold increase in the application-focused forecasting literature between the 1990s and the current decade, with a clear rise of quantitative, data-driven forecasting models. Comparative studies of quantitative methods and human judgement show that (1) neither method is universally superior, and (2) the better method varies as a function of factors such as availability, quality, extent and format of data, suggesting that (3) the two approaches can complement each other to yield more accurate and resilient models. We also identify four research thrusts in the human/machine-forecasting literature: (i) the choice of the appropriate quantitative model, (ii) the nature of the interaction between quantitative models and human judgement, (iii) the training and incentivization of human forecasters, and (iv) the combination of multiple forecasts (both algorithmic and human) into one. This review surveys current research in all four areas and argues that future research in the field of human/machine forecasting needs to consider all of them when investigating predictive performance. We also address some of the ethical dilemmas that might arise due to the combination of quantitative models with human judgement.

show abstract

“…Previous research on advice-taking explored various cues that people use to infer competence [ 31 , 32 ]. In this paper, we focused on two of them: expertise and the outcome of the decision.…”

Section: Introductionmentioning

confidence: 99%

“…From early childhood human beings are attuned to others’ expertise [ 31 – 39 ]. “Says who?” is what we immediately think when exposed to someone else’s opinion [ 40 ].…”

Section: Introductionmentioning

confidence: 99%

Evidence for arrogance: On the relative importance of expertise, outcome, and manner

et al. 2017

View full text Add to dashboard Cite

Arrogant behavior is as old as human nature. Nonetheless, the factors that cause people to be perceived as arrogant have received very little research attention. In this paper, we focused on a typical manifestation of arrogance: dismissive behavior. In particular, we explored the conditions under which a person who dismissed advice would be perceived as arrogant. We examined two factors: the advisee’s competence, and the manner in which he or she dismissed the advice. The effect of the advisee’s competence was tested by manipulating two competence cues: relative expertise, and the outcome of the advice dismissal (i.e., whether the advisee was right or wrong). In six studies (N = 1304), participants made arrogance judgments about protagonists who dismissed the advice of another person while the advisees’ relative expertise (compared to the advisor), their eventual correctness, and the manner of their dismissal were manipulated in between-participant designs. Across various types of decisions and advisee-advisor relationships, the results show that less expert, less correct, and ruder advisees are perceived as more arrogant. We also find that outcome trumps expertise, and manner trumps both expertise and outcomes. In two additional studies (N = 101), we examined people’s naïve theories about the relative importance of the aforementioned arrogance cues. These studies showed that people overestimate the role of expertise information as compared to the role of interpersonal manner and outcomes. Thus, our results suggest that people may commit arrogant faux pas because they erroneously expect that their expertise will justify their dismissive behavior.

show abstract

Evaluating expert advice in forecasting: Users’ reactions to presumed vs. experienced credibility

Cited by 19 publications

References 71 publications

Preference for human or algorithmic forecasting advice does not predict if and how it is used

Preference for human or algorithmic forecasting advice does not predict if and how it is used

A survey of human judgement and quantitative forecasting methods

Evidence for arrogance: On the relative importance of expertise, outcome, and manner

Contact Info

Product

Resources

About