Facilitating sender-receiver agreement in communicated probabilities: Is it best to use words, numbers or both?

Abstract

Organizations tasked with communicating expert judgments couched in uncertainty often use numerically bounded linguistic probability schemes to standardize the meaning of verbal probabilities. An experiment (N = 1,202) was conducted to ascertain whether agreement with such a scheme was better when probabilities were presented verbally, numerically or in a combined “verbal + numeric” format. Across three agreement measures, the numeric and combined formats outperformed the verbal format and also yielded better discrimination between low and high probabilities and were less susceptible to the fifty-fifty blip phenomenon. The combined format did not confer any advantage over the purely numeric format. The findings indicate that numerically bounded linguistic probability schemes are an ineffective means of communicating information about probabilities to others and they call into question recommendations for use of the combined format for delivering such schemes.

Communicating uncertainty in national security intelligence: Expert and nonexpert interpretations of and preferences for verbal and numeric formats

“…Notwithstanding the widespread adoption of NBLP schemes by organizations tasked with producing expert probabilistic judgments, several studies show that people's interpretations often do not conform to them, even when the stipulated numeric ranges are embedded in-text alongside verbal probabilities (e.g., x is likely [60%−80%]; Budescu et al, 2009;Budescu et al, 2012;Budescu et al, 2014). Recent studies have generalized these findings to NBLP schemes used by the intelligence community (Ho et al, 2015;Mandel & Irwin, 2021a, 2021bWintle et al, 2019).…”

“…Comparing expert and nonexpert samples allowed us to assess generalizability across samples. Moreover, despite the lack of empirical support for NBLP schemes cited earlier and among growing calls for intelligence organizations to use numeric probabilities (Barnes, 2016; Dhami & Mandel, 2021; Friedman, 2019; Mandel & Irwin, 2021a, 2021c; Mandel, Wallsten, et al., 2021), intelligence community (and other) skeptics may question the generalizability of research conducted with samples of nonexperts or experts from different fields. Skeptics may also believe that numeric probabilities convey bogus “scientific” precision and that intelligence consumers prefer to receive verbal probabilities (contrary to the aforementioned preference paradox).…”

“…We also examined whether the probability format (i.e., verbal or numeric) used in intelligence estimates affected judgments of praise or blame. This is an important consideration because the intelligence community's institutional preference for verbal uncertainty expressions has been framed as a defensive strategy on the part of blame‐avoidant analysts (Beyth‐Marom, 1982; Gentry, 2017; Mandel & Irwin, 2021a; Tetlock & Mellers, 2011). However, recent studies show that erroneous judgments sometimes garner less criticism when communicated numerically than verbally (Collins & Mandel, 2019; Jenkins & Harris, 2021; Jenkins et al., 2019).…”

“…Past research has shown that higher numeracy facilitates probability assessment and risk estimation, whereas lower numeracy predicts greater trust of verbal than numerical information and a heightened susceptibility to presentation effects (Lipkus & Peters, 2009; Reyna et al., 2009). Numeracy is also positively associated with adherence to NBLP schemes promulgated by intelligence organizations (Wintle et al., 2019; Mandel & Irwin, 2021a). We also measured actively open‐minded thinking (AOT), which reflects an individual's openness to alternative opinions and contradictory evidence (Baron et al., 2015).…”

The effects of communicating scientific uncertainty on trust and decision making in a public health context

“…Uncertainty, for instance, around an estimate of an effect -'direct' uncertainty (van der Bles et al, 2019) -is known to play a role in how people perceive information and use it in their decision making (Gustafson & Rice, 2020). The way in which direct uncertainty around a number is communicated, such as verbally or numerically for instance, can lead to different interpretations of the information, and much research has shed light on these relationships (Dhami & Mandel, 2021;Kause et al, 2021;Mandel & Irwin, 2021;van der Bles et al, 2020). Additionally, research has shown that the format in which such direct uncertainty (e.g., probabilistic predictions) is communicated can also affect the perceived credibility of the communicator (Collins & Mandel, 2019;Jenkins et al, 2017;Jenkins et al, 2018;Jenkins & Harris, 2021).…”