Framing and Contextualizing Information Requests: Problem Formulation as Part of the Intelligence Analysis Process

Abstract

Naturalistic decision-making studies of intelligence analysis have generally focused on information search, collection, and synthesis processes, deemphasizing the initial "problem formulation" phase, in which analysts interpret and contextualize the information request to determine which information to collect. We present the results of two studies focusing on this phase. In the first study, we performed a cognitive task analysis via semistructured interviews with 22 active-duty U.S. Army intelligence analysts to uncover factors that arise in operational environments that complicate problem formulation. The factors discovered (e.g., vague and/or overly narrow intelligence requests) led to a second study probing 6 active-duty U.S. Army intelligence analysts' cognitive strategies with a "think-aloud" protocol as they interpreted and evaluated representative information requests. The study revealed that analysts actively interpret and contextualize an information request. The analysts reframed and broadened the request so that they could respond meaningfully to the underlying intent, then used contextual cues and metainformation to determine the most useful collectors and how effectively the request could be answered in the time allotted. We discuss these results and their implications for both the cognitive modeling of intelligence analysis and the development of training and decision aids for more effective framing and contextualization of information requests.

Supporting Representation Management in Intelligence Analysis through Automated Decision Aids

“…This failure can be linked to two patterns in clumsy automation support. First, many software tools attempt to supplant analyst reasoning, offloading analytical tasks without considering the context needed for the analyst to reintegrate received results Roth et al, 2010;Woods et al, 2002). These tools emphasize the narrowing of the information space provided to the analyst (e.g., through traditional filter and bin approaches, outmoded data-centric use cases), and often ignore the critical need for broadening checks that support analyst ability to constantly explore a range of data potentially related to a current problem (Zelik, Patterson, and Woods, 2007).…”

“…The model presents hypotheses about how new information technology tools can effectively support the cognitive work underlying abductive inferential analysis. A key artifact from this research base has been the development and continued refinement of the goals and sub-goals in the support model that characterizes the iterative and convergent sense-making processes with regard to four primary interacting cognitive functions (Elm et al, 2005;Roth et al, 2010;Voshell et al, 2012):…”

“…One of the most difficult challenges for integrating these new technologies into effective information analysis support systems involves the fundamental mismatch between the increasing amounts of data availability and the lack of useful automation to support human analysis of this information (Woods, Patterson, and Roth, 1998). The growing disparity between data availability and analyst support increases vulnerability to intelligence failures and can delay exploitation of critical information-a consistent data overload pattern found in the Cognitive Systems Engineering (CSE) literature (e.g., Roth et al, 2010;Woods, Patterson, & Roth, 2002). Today, automation support design and interaction approaches are not sufficiently integrated into analyst workflows, and system developers often erroneously attempt to replace human interpretation rather than aid it as part of a constructive joint human-automation analytic process.…”

Common ground in collaborative intelligence analysis: an empirical study

“…Likewise, the negotiation of common ground is critical to advancing the cognitive and communicative processes needed to support those actions. In point of fact, a shared understanding of the intelligence task, it's context and the motivation behind it, all of which are significant for interpreting and framing of the intelligence request is common ground (Roth et al, 2010).…”

Combining cognitive engineering and information fusion architectures to build effective joint systems

“…One of the most challenging aspects for introducing new support systems for effective information analysis involves closing the gap between the increasing amounts of data availability and the lack of useful automation to support human analysis of this information [1], [2] . Elm et al's [3] hypothesized support model of information analysis-extended by Roth et al [4] -serves as an evolving framework to identify information needs and decision-making requirements for designing effective intelligence analysis tools and systems . Based on findings to support coping with data overload [5] , the support model is built around four high-level functions-Framing, Down Collect, Conflict and Corroboration, and Hypothesis Exploration [3], [4], [6] -that interact as part of an iterative broadening, narrowing, and convergence process .…”