Insider Threat Detection in PRODIGAL

Goldberg, Henry G.; Young, William T.; Reardon, Matthew; Phillips, Brian J.; Senator, Ted E.

doi:10.24251/hicss.2017.320

Cited by 16 publications

(7 citation statements)

References 22 publications

(41 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many insider threat detection systems 16–19 were stemmed from DARPA's project ADAMS, 20 which aims to identify patterns and anomalies in very large datasets to combat insider threats. In Goldberg et al 18 and Senator et al, 19 various anomaly detection algorithms, including hidden Markov model (HMM) and Gaussian mixture model, were employed on user activity log data for identifying indicators of insider threats. Eldardiry et al proposed approaches employing hybrid of anomaly detectors on combined information from multiple domains (user activities) to detect blend‐in anomalies and unusual change anomalies 16 .…”

Section: Related Workmentioning

confidence: 99%

Exploring anomalous behaviour detection and classification for insider threat identification

Zincir-Heywood

2020

Int J Network Mgmt

View full text Add to dashboard Cite

Summary Recently, malicious insider threats represent one of the most damaging threats to companies and government agencies. Insider threat detection is a highly skewed data analysis problem, where the huge class imbalance makes the adaptation of learning algorithms to the real‐world context very difficult. This study proposes a new system for user‐centred machine learning‐based anomaly behaviour and insider threat detection on multiple data granularity levels. System evaluations and analysis are performed not only on individual data instances but also on normal and malicious users. Our results show that the proposed system, which is a combination of unsupervised anomaly detection and supervised machine learning methods, can learn from unlabelled data and a very small amount of labelled data. Furthermore, it can generalize to bigger datasets for detecting anomalous behaviours and unseen malicious insiders with a high detection and a low false‐positive rate.

show abstract

Section: Related Workmentioning

confidence: 99%

Exploring anomalous behaviour detection and classification for insider threat identification

Zincir-Heywood

2020

Int J Network Mgmt

View full text Add to dashboard Cite

show abstract

“…The CMU-CERT data sets are synthetic insider threat data sets generated by the CERT Division at Carnegie Mellon University [8,15]. CMU-CERT data repository is the only one available for insider threat scenarios (5 scenarios) and has recently become the evaluation data repository for researchers addressing the insider threat problem [5,16,41].…”

Section: Description Of the Datasetmentioning

confidence: 99%

“…This sheds light on the importance of the FP measure to address the shortcoming of the high number of false alarms (FPs). Furthermore, some approaches were validated in terms of: TP measure [31]; F1 measure [4,33]; AUC measure [9,11,16]; precision and recall [24,29]; accuracy [24,35]; and others.…”

Section: Evaluation Measuresmentioning

confidence: 99%

Data Stream Clustering for Real-Time Anomaly Detection: An Application to Insider Threats

Haidar

Gaber

2018

Clustering Methods for Big Data Analytics

View full text Add to dashboard Cite

Insider threat detection is an emergent concern for academia, industries, and governments due to the growing number of insider incidents in recent years. The continuous streaming of unbounded data coming from various sources in an organisation, typically in a high velocity, leads to a typical Big Data computational problem. The malicious insider threat refers to anomalous behaviour(s) (outliers) that deviate from the normal baseline of a data stream. The absence of previously logged activities executed by users shapes the insider threat detection mechanism into an unsupervised anomaly detection approach over a data stream. A common shortcoming in the existing data mining approaches to detect insider threats is the high number of false alarms/positives (FPs). To handle the big data issue and to address the shortcoming, we propose a streaming anomaly detection approach, namely Ensemble of Random subspace Anomaly detectors In Data Streams (E-RAIDS), for insider threat detection. E-RAIDS learns an ensemble of p established outlier detection techniques (Micro-cluster-based Continuous Outlier Detection-MCODor Anytime Outlier Detection-AnyOut-) which employ clustering over continuous data streams. Each model of the p models learns from a random feature subspace to detect local outliers, which might not be detected over the whole feature space. E-RAIDS introduces an aggregate component that combines the results from the p feature subspaces, in order to confirm whether to generate an alarm at each window iteration. The merit of E-RAIDS is that it defines a survival factor and a vote factor to address the shortcoming of high number of FPs. Experiments on E-RAIDS-MCOD and E-RAIDS-AnyOut are carried out, on synthetic data sets including malicious insider threat scenarios generated at Carnegie Mellon University, to test the effectiveness of voting feature subspaces, and the capability to detect (more than one)-behaviour-all-threat in real-time. The results show that E-RAIDS

show abstract

“…A further recent unsupervised ensemble-based anomaly detection system named PRODIGAL was presented in [11]; a result of five years work on the insider threat detection problem [12], [13], [14]. iForest is configured as one of the user-day detectors in PRODIGAL to detect complex insider threat scenarios in real user activities.…”

Section: Related Workmentioning

confidence: 99%

“…In this work, we utilised two highly performing anomaly detection algorithms: ocsvm and iForest, as base algorithms in the proposed framework to detect malicious insider threats. We adopted ocsvm and iForest, because each method has been utilised for insider threat detection, either as a base algorithm for the proposed approaches [5], [9], [10], [11], or as a benchmark against which the performance of a deep learning approach was compared to its performance [16]. a) Clustering Component: Malicious insiders have authorised access to the network, system, and data, and are aware of the system management and security policies.…”

Section: B Anomaly Detection Frameworkmentioning

confidence: 99%

Adaptive One-Class Ensemble-based Anomaly Detection: An Application to Insider Threats

Haidar¹,

Gaber²

2018

2018 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

The malicious insider threat is getting increased concern by organisations, due to the continuously growing number of insider incidents. The absence of previously logged insider threats shapes the insider threat detection mechanism into a one-class anomaly detection approach. A common shortcoming in the existing data mining approaches to detect insider threats is the high number of False Positives (FP) (i.e. normal behaviour predicted as anomalous). To address this shortcoming, in this paper, we propose an anomaly detection framework with two components: one-class modelling component, and progressive update component. To allow the detection of anomalous instances that have a high resemblance with normal instances, the oneclass modelling component applies class decomposition on normal class data to create k clusters, then trains an ensemble of k base anomaly detection algorithms (One-class Support Vector Machine or Isolation Forest), having the data in each cluster used to construct one of the k base models. The progressive update component updates each of the k models with sequentially acquired FP chunks; segments of a predetermined capacity of FPs. It includes an oversampling method to generate artificial samples for FPs per chunk, then retrains each model and adapts the decision boundary, with the aim to reduce the number of future FPs. A variety of experiments is carried out, on synthetic data sets generated at Carnegie Mellon University, to test the effectiveness of the proposed framework and its components. The results show that the proposed framework reports the highest F1 measure and less number of FPs compared to the base algorithms, as well as it attains to detect all the insider threats in the data sets.

show abstract

Insider Threat Detection in PRODIGAL

Cited by 16 publications

References 22 publications

Exploring anomalous behaviour detection and classification for insider threat identification

Exploring anomalous behaviour detection and classification for insider threat identification

Data Stream Clustering for Real-Time Anomaly Detection: An Application to Insider Threats

Adaptive One-Class Ensemble-based Anomaly Detection: An Application to Insider Threats

Contact Info

Product

Resources

About