Uncertainty handling in estimative intelligence – challenges and requirements from both analyst and consumer perspectives

Abstract

Important assessments of events and activities relating to military, terrorist and hybrid adversaries, and the intentions of foreign governments, are made every day, usually involving subjective or 'estimative' probabilities and an associated level of confidence. The way in which these uncertainties are assessed and communicated can potentially have enormous impact and consequences. Challenges are reinforced by increasingly complex intelligence problems for which the contemporary analytic paradigm is not tailored to cope. It is important to better understand how defence intelligence analysts and consumers handle uncertainty in their assessment and decision support activities, and what challenges and requirements they face in doing so. This is mainly achieved by the use of semi-structured interviews with a sample of very senior consumers of military intelligence (mostly Flag Officers of the Norwegian Armed Forces) and focus group interviews with groups of Norwegian intelligence analysts. In general, respondents found it difficult or challenging to conceptualize uncertainty analytically. This has implications for the communication of uncertainty and its use in decision-making within the current framework. Secondly, respondents were receptive to suggested potential improvements to the existing framework. One such suggestion involved a differentiated framework, offering different levels of uncertainty resolution in different situations, although none of the respondents had any experience of such a framework for assessing or communicating uncertainty. We conclude with some recommendations to improve the process of uncertainty and risk communication in this important and consequential application area. Having particular implications for policy, we recommend that analysts follow a differentiated approach in handling different situations and problems comprising uncertainty, rather than pursuing a standard solution as is current practice.