Design and Analysis of a Dynamically Configured Log-based Distributed Security Event Detection Methodology

Abstract

Military and defense organizations rely upon the security of data stored in, and communicated through, their cyber infrastructure to fulfill their mission objectives. It is essential to identify threats to the cyber infrastructure in a timely manner, so that mission risks can be recognized and mitigated. Centralized event logging and correlation is a proven method for identifying threats to cyber resources. However, centralized event logging is inflexible and does not scale well, because it consumes excessive network bandwidth and imposes significant storage and processing requirements on the central event log server. In this paper, we present a flexible, distributed event correlation system designed to overcome these limitations by distributing the event correlation workload across the network of event-producing systems. To demonstrate the utility of the methodology, we model and simulate centralized, decentralized, and hybrid log analysis environments over three accountability levels and compare their performance in terms of detection capability, network bandwidth utilization, database query efficiency, and configurability. The results show that when compared to centralized event correlation, dynamically configured distributed event correlation provides increased flexibility, a significant reduction in network traffic in low and medium accountability environments, and a decrease in database query execution time in the high-accountability case.

Topological Data Analysis for Enhancing Embedded Analytics for Enterprise Cyber Log Analysis and Forensics

“…To solve this problem, various analytical methods have been proposed to analyze logs [24]. Log analytics methods include both high level approaches, such as anomaly detection [25] [26], text analytics [27], dynamic rule creation [28], and event correlation [24], to the application of specific algorithms such as support vector machines [29], random forests [30], principal component analysis (PCA) and factor analysis (FA) [3]. However, analytics by themselves can be brittle.…”

An approach to the correlation of security events based on machine learning techniques

“…In a military and defense context, Grimaila et al [19] also propose a distributed approach to security event correlation in order to "identify potential threats in a timely manner". Rieke and Stoynova [20] present a blueprint of an archtitecture for predictive security analysis that uses process models in extension to security policies and models.…”

Cyber anomaly detection: Using tabulated vectors and embedded analytics for efficient data mining

“…Related research, c.f. Lazarevic et al, 18 Denning, 19 Garc ıa-Teodoro et al, 20 Grimaila et al, 21 Moore et al, 22 Dube et al, 23 Shilland, 24 Shen et al, 25 Stewart et al 26 has focused on anomaly detection at the device/software level, with little 21,[27][28][29][30][31][32] exploration into anomaly detection in the log files generated from the preexisting devices or software.…”