We conducted two studies that investigated display characteristics related to color (hue, saturation, brightness, and transparency) and contrast with a background for displaying information qualifiers (termed meta-information) such as uncertainty, age, and source quality. Level of detail (or granularity) of the meta-information and task demands were also manipulated. Participants were asked to rank and rate colored regions overlaid on different map backgrounds based on the level of meta-information the regions displayed. Results from Study 1 indicated that participants could appropriately rank and rate levels of meta-information across saturation, brightness, and transparency conditions, and results from Study 1 and Study 2 showed that the natural direction of ordering is complex and dependent on the relevance of different information to the task and the contrast of the overlay region with the background.

Naturalistic decision-making studies of intelligence analysis have generally focused on information search, collection, and synthesis processes, deemphasizing the initial "problem formulation" phase, in which analysts interpret and contextualize the information request to determine which information to collect. We present the results of two studies focusing on this phase. In the first study, we performed a cognitive task analysis via semistructured interviews with 22 active-duty U.S. Army intelligence analysts to uncover factors that arise in operational environments that complicate problem formulation. The factors discovered (e.g., vague and/or overly narrow intelligence requests) led to a second study probing 6 active-duty U.S. Army intelligence analysts' cognitive strategies with a "think-aloud" protocol as they interpreted and evaluated representative information requests. The study revealed that analysts actively interpret and contextualize an information request. The analysts reframed and broadened the request so that they could respond meaningfully to the underlying intent, then used contextual cues and metainformation to determine the most useful collectors and how effectively the request could be answered in the time allotted. We discuss these results and their implications for both the cognitive modeling of intelligence analysis and the development of training and decision aids for more effective framing and contextualization of information requests.

No abstract

Information, as well as its qualifiers, or metainformation, forms the basis of human decisionmaking. Modeling human reasoning therefore requires the development of representations of both information and meta-information. However, while existing models and modeling approaches may include computational technologies that support meta-information analysis, they generally neglect its role in human reasoning. Herein, we describe the application of Bayesian Belief Networks to model how humans calculate, aggregate, and reason about metainformation when making decisions.

Supporting complex decision making requires conveying relevant information characteristics or qualifiers. The authors tested transparency and numeric annotation for displaying uncertainty about object identity. Participants performed a "missile defense" game in which they decided whether to destroy moving objects (which were either threatening missiles or nonthreatening birds and planes) before they reached a city. Participants were provided with uncertain information about the objects' classifications. Uncertainty was represented through the transparency of icons representing the objects and/or with numeric annotations. Three display methods were created. Icons represented the most likely object classification (with solid icons), the most likely object classification (with icons whose transparency represented the level of uncertainty), or the probability that the icon was a missile (with transparency). In a fourth condition, participants could choose among the representations. Icons either were or were not annotated with numeric probability labels. Task performance was highest when participants could toggle the displays, with little effect of numeric annotation. In conditions in which probabilities were available graphically or numerically, participants chose to engage objects when they were farther from the city and had a lower probability of being a missile. Results provided continued support for the use of graphical uncertainty representations, even when numeric representations are present.