Towards an evidence-based approach to communicating uncertainty in intelligence analysis

Dhami, Mandeep K.

doi:10.1080/02684527.2017.1394252

Cited by 24 publications

(34 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, given the vagueness inherent in verbal probabilities, methods have been developed for translating verbal expressions in a communicator's lexicon to equivalently ranked expressions in a listener's lexicon (Dhami & Wallsten, 2005;Karelitz & Budescu, 2004). Another approach is to establish a standardized lexicon of verbal probabilities, and indeed, many intelligence organizations have done so (e.g., Barnes, 2015;Dhami, 2013). Behavioral research methods have been used to develop standardized lexicons that people are more likely to use (Ho, Budescu, Dhami, & Mandel, in press).…”

Section: Decision Research On Communicating Uncertaintymentioning

confidence: 99%

Improving Intelligence Analysis With Decision Science

Dhami

Mandel

Mellers

et al. 2015

Perspect Psychol Sci

Self Cite

112

View full text Add to dashboard Cite

Intelligence analysis plays a vital role in policy decision making. Key functions of intelligence analysis include accurately forecasting significant events, appropriately characterizing the uncertainties inherent in such forecasts, and effectively communicating those probabilistic forecasts to stakeholders. We review decision research on probabilistic forecasting and uncertainty communication, drawing attention to findings that could be used to reform intelligence processes and contribute to more effective intelligence oversight. We recommend that the intelligence community (IC) regularly and quantitatively monitor its forecasting accuracy to better understand how well it is achieving its functions. We also recommend that the IC use decision science to improve these functions (namely, forecasting and communication of intelligence estimates made under conditions of uncertainty). In the case of forecasting, decision research offers suggestions for improvement that involve interventions on data (e.g., transforming forecasts to debias them) and behavior (e.g., via selection, training, and effective team structuring). In the case of uncertainty communication, the literature suggests that current intelligence procedures, which emphasize the use of verbal probabilities, are ineffective. The IC should, therefore, leverage research that points to ways in which verbal probability use may be improved as well as exploring the use of numerical probabilities wherever feasible.

show abstract

Section: Decision Research On Communicating Uncertaintymentioning

confidence: 99%

Improving Intelligence Analysis With Decision Science

Dhami

Mandel

Mellers

et al. 2015

Perspect Psychol Sci

Self Cite

112

View full text Add to dashboard Cite

show abstract

“…However, research has shown that phrases with negative directionality are interpreted with greater variability than positively worded phrases [20]. This is supported by evidence from research on intelligence analysts [19], [21], [22].…”

Section: Uk and Us Policies For Communicating Probability In Intelligmentioning

confidence: 88%

“…It has been argued that providing synonyms allows for stylistic expression in intelligence reports [13]. However, some evidence suggests that synonyms in the lexicons may not be interchangeable in the minds of analysts [10], [21], [22]. Conversely, recipients of communications might treat terms that are differentiated in the lexicons as near synonymous.…”

Section: Uk and Us Policies For Communicating Probability In Intelligmentioning

confidence: 99%

“…Barnes [13] "banned" the use of the phrase 'realistic possibility' in his Canadian probability lexicon because of his belief that it lacked precision (see also Kent's 1964 discussion of weasel words [14]). Indeed, studies reveal that US and UK analysts do not normally use this phrase to express probability [21], [22].…”

Section: Uk and Us Policies For Communicating Probability In Intelligmentioning

confidence: 99%

“…Partly in order to overcome this limitation, other studies have used quantitative methods [10], [19], [21], [22], that elicit people's lexicons for communicating probability and measure how people numerically interpret linguistic probabilities (for details of some specific methods used, see Refs. [33], [46], [47]).…”

mentioning

confidence: 99%

See 2 more Smart Citations

UK and US policies for communicating probability in intelligence analysis: A review

Mk¹,

Mandel²

2020

Preprint

View full text Add to dashboard Cite

show abstract

Arithmetic computation with probability words and numbers

Mandel

Dhami

Tran

et al. 2021

Behavioral Decision Making

Self Cite

View full text Add to dashboard Cite

Probability information is regularly communicated to experts who must fuse multiple estimates to support decision-making. Such information is often communicated verbally (e.g., "likely") rather than with precise numeric (point) values (e.g., ".75"), yet people are not taught to perform arithmetic on verbal probabilities. We hypothesized that the accuracy and logical coherence of averaging and multiplying probabilities will be poorer when individuals receive probability information in verbal rather than numerical point format. In four experiments (N = 213, 201, 26, and 343, respectively), we manipulated probability communication format between-subjects. Participants averaged and multiplied sets of four probabilities. Across experiments, arithmetic accuracy and coherence was significantly better with point than with verbal probabilities. These findings generalized between expert (intelligence analysts) and non-expert samples and when controlling for calculator use. Experiment 4 revealed an important qualification: whereas accuracy and coherence were better among participants presented with point probabilities than with verbal probabilities, imprecise numeric probability ranges (e.g., ".70 to .80") afforded no computational advantage over verbal probabilities. Experiment 4 also revealed that the advantage of the point over the verbal format is partially mediated by strategy use. Participants presented with point estimates are more likely to use mental computation than guesswork, and mental computation was found to be associated with better accuracy. Our findings suggest that where computation is important, probability information should be communicated to end users with precise numeric probabilities.

show abstract

Towards an evidence-based approach to communicating uncertainty in intelligence analysis

Cited by 24 publications

References 39 publications

Improving Intelligence Analysis With Decision Science

Improving Intelligence Analysis With Decision Science

UK and US policies for communicating probability in intelligence analysis: A review

Arithmetic computation with probability words and numbers

Contact Info

Product

Resources

About