Detecting Unknown Insider Threat Scenarios

Abstract

Abstract-This paper reports results from a set of experiments that evaluate an insider threat detection prototype on its ability to detect scenarios that have not previously been seen or contemplated by the developers of the system. We show the ability to detect a large variety of insider threat scenario instances imbedded in real data with no prior knowledge of what scenarios are present or when they occur. We report results of an ensemble-based, unsupervised technique for detecting potential insider threat instances over eight months of real monitored computer usage activity augmented with independently developed, unknown but realistic, insider threat scenarios that robustly achieves results within 5% of the best individual detectors identified after the fact. We explore factors that contribute to the success of the ensemble method, such as the number and variety of unsupervised detectors and the use of prior knowledge encoded in scenario-based detectors designed for known activity patterns. We report results over the entire period of the ensemble approach and of ablation experiments that remove the scenario-based detectors.

Insider Threat Detection in PRODIGAL

“…Experiments and results included in this paper extend previously reported results ( [21], [24], [25]) to cover 16 months of data from September 2012 through February 2014. Testing on the additional eight months included a number of new Red Team (RT) scenarios, new detection algorithms, and improved versions of existing detectors.…”

“…Since this paper continues and extends experiments previously reported in [25], we have repeated some introductory material and descriptions of our approach -especially regarding metrics, the test data, and the ensemble approach. While the previous paper looked at results from data collected over eight months, we cover results from 16 months, including a number of novel inserted target scenarios.…”

“…Anomaly detector ensemble methods combine the results (i.e., scores) from multiple detectors in a way that is analogous to how classifier ensembles combine predictions from multiple classifiers [8]. The following description of our ensemble approach is taken from [25]. We have not significantly altered the algorithm for the experiments reported here.…”

“…We extend preliminary results (from [25]) to an additional eight months, including 27 additional threat data sets comprising 15 new Red Team scenarios. The average AUC achieved by the ensemble over the new data is identical to that over the initial data, at 0.85, while the average "best" detector AUC only improved slightly from 0.88 to 0.89.…”

“…To illustrate PRODIGAL's ability to detect such complex scenarios, we repeat a discussion of one such target scenario inserted in multiple data set by the Red Team as described in [25].…”