We retrospectively explore the effectiveness of various probabilistic opinion pools against a set of insider threat detection modeling data from a recently completed, multiyear, sponsored research effort. We explored four opinion pools: the linear opinion pool (likely the most popular), the beta-transformed linear opinion pool, the geometric opinion pool, and a multiplicative method based on odds called Bordley’s formula. The data for our study came from our recent work in the inference of insider threats for our research sponsor. In this work, we created a multimodeling inference enterprise modeling (MIEM) process to either predict threats within a population or, given the threats, predict how well the enterprise system can detect those threats. As part of larger research challenges designed by the research sponsor, we applied the MIEM process quarterly to respond to a sequence of varying challenge problems (CPs). Via MIEM, we developed multiple, independent computation forecast models. These models generated certainty intervals to answer CP questions. These intervals were fused into a single interval for each question via an expert panel prior to submission. The sponsors scored the responses against ground truth. In this paper, we (a) ask which pooling functions work best on these data and consider why, and (b) compare this performance to the actual submissions to determine if one of the pooling functions performed better than our judgment-based fusion.
We define an innovation process that was designed for the provision of analytic services and software products. Our iterative process, which is based on design thinking, captures the need to both create value and accelerate value creation. The four steps in our iterative process include illuminate, ideate, create, and evaluate. We present a case study for which we used this process in the successful creation of a conversational non‐player character within a virtual training environment for the Army.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.