Recent research suggests that communicating probabilities numerically rather than verbally benefits forecasters’ credibility. In two experiments, we tested the reproducibility of this communication-format effect. The effect was replicated under comparable conditions (low probability, inaccurate forecasts), but it was reversed for low-probability accurate forecasts and eliminated for high-probability forecasts. Experiment 2 further showed that verbal probabilities convey implicit recommendations more clearly than probability information, whereas numeric probabilities do the opposite. Descriptively, the findings indicate that the effect of probability words versus numbers on credibility depends on how these formats convey directionality differently, how directionality implies recommendations even when none are explicitly given, and how such recommendations correspond with outcomes. Prescriptively, we propose that experts distinguish forecasts from advice, using numeric probabilities for the former and well-reasoned arguments for the latter.

Three experiments (N = 854) examined the effect of a four-step elicitation method used in several expert elicitation studies on judgment accuracy. Participants made judgments about topics that were either searchable or unsearchable online using one of two order variations of the four-step procedure. One group of participants provided their best judgment (one step) prior to constructing an interval (i.e., lower bound, upper bound, and a confidence rating that the correct value fell in the range provided), whereas another group of participants provided their best judgment last after the three-step confidence interval was constructed. The overall effect of this elicitation method was not significant. The accuracy of best estimates was not improved by prior interval construction, and the accuracy of confidence intervals was not affected by elicitation order either. The coherence of participants’ judgments, however, stably predicted more accurate best estimates and also wider credible intervals.

Individuals often assess themselves as being less susceptible to common biases compared to others. This bias blind spot (BBS) is thought to represent a metacognitive error. In this research, we tested three explanations for the effect: The cognitive sophistication hypothesis posits that individuals who display the BBS more strongly are actually less biased than others. The introspection bias hypothesis posits that the BBS occurs because people rely on introspection more when assessing themselves compared to others. The conversational processes hypothesis posits that the effect is largely a consequence of the pragmatic aspects of the experimental situation rather than true metacognitive error. In two experiments (N = 1057) examining 18 social/motivational and cognitive biases, there was strong evidence of the BBS. Among the three hypotheses examined, the conversational processes hypothesis attracted the greatest support, thus raising questions about the extent to which the BBS is a metacognitive effect.

This is a pre-print of a chapter intended for publication in the book "JUDGMENT IN PREDICTIVE ANALYTICS" (Ed. Matthias Seifert, Ph.D.). The chapter represents a primer and review of various performance-weighted aggregation techniques, including: history-based weighting methods, disposition-based weighting methods, and coherence-based weighting methods. Various formulas for applying these measures of 'performance' in practical weighted aggregation schemes are also discussed.

Decision-makers in diverse fields ranging from medicine to law to public policy depend on the clear and unbiased communication of probabilities. Expert judges often express probabilities verbally (e.g., unlikely) rather than numerically (e.g., 25% chance). Researchers have criticized the practice because verbal probabilities are vague and imprecise. However, little work has investigated the pragmatic consequences of their usage. In three online experiments (Ns = 201, 439, 435), we showed that the directionality of verbal probabilities leaked information about experts’ expectations. Receivers interpret these inferred expectations as recommendations. The inferred recommendation, in turn, influences their decisions. Compared to numeric probabilities, the directionality of verbal probabilities makes their interpretation more ambiguous due to pragmatic implicatures. We discuss how senders could improve communication fidelity by using numeric probabilities as a principal form of communicating probabilistic information, as their directionality and interpretation are less ambiguous and more consistent.