Cyber Intrusion Detection using Natural Language Processing on Windows Event Logs

Steverson, Kai; Carlin, Caleb; Mullin, Jonathan M.; Ahiskali, Metin

doi:10.1109/icmcis52405.2021.9486307

Cited by 6 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The AAU_MalData dataset is divided into two subsets, AAU_MalData_Pure and AAU_MalData_Contagious; the first one corresponds to the regular operation of the Windows OS, while the latter to the analysis of 2,800 malware samples of Trojans and Backdoors type with the help of the proposed framework. Such dataset can be utilized, for instance, for the training of machine learning algorithms as suggested in [25] for advancing antimalware research.…”

Section: Discussionmentioning

confidence: 99%

Redefining Malware Sandboxing: Enhancing Analysis Through Sysmon and ELK Integration

Mahmoud,

Anagnostopoulos,

Pastrana

et al. 2024

IEEE Access

View full text Add to dashboard Cite

In cybersecurity, adversaries employ a myriad of tactics to evade detection and breach defenses. Malware remains a formidable weapon in their arsenal. To counter this threat, researchers unceasingly pursue dynamic analysis, which aims to comprehend and thwart established malware strains. This paper introduces an innovative methodology for dynamic malware analysis while critically evaluating prevailing technologies and their limitations. The proposed approach hinges on harnessing the capabilities of an open-source Security Information and Event Management (SIEM) toolset, namely the Elastic-Stack. This toolset is utilized to capture, structure, and analyze the behavioral patterns of malware without relying on any pre-existing sandbox framework. This augmentation facilitates a profound understanding of the activities exhibited by malware samples. With the help of the proposed ecosystem, we compile the AAU_MalData dataset that encompasses distinctly the benign and malicious behavior. Specifically, we analyzed the behavior of the 2,800 malware within a realistic network topology and systematically collected Windows event logs, which serve as a comprehensive record of the malware's actions. These event logs are precious as they are organized in a timestamped format, providing a chronological list of system activities, such as event descriptions, process and file details, and registry modifications. These are pivotal in comprehending the malware's functionality and repercussions on the compromised system. Furthermore, by incorporating the MITRE ATT&CK framework, we leveraged the event logs to correlate the malware's mode of operation, delve into its Command and Control operations, and investigate its persistence mechanisms, enabling a structured and practical approach to malware analysis. The AAU_MalData dataset, organized in a JSON format data structure, is offered to the research community first as a proof of concept to demonstrate the toolset's feasibility for dynamic malware analysis and second as a potential training ground for anti-malware mechanisms based on host and network Indicators of Compromise (IoCs).

show abstract

Section: Discussionmentioning

confidence: 99%

Redefining Malware Sandboxing: Enhancing Analysis Through Sysmon and ELK Integration

Mahmoud,

Anagnostopoulos,

Pastrana

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In addition, traditional machine learning methods cannot efectively address the heterogeneity and evolution of logs, making the accuracy of anomaly detection based on traditional machine learning methods not very high. With the rapid development of deep learning and natural language processing, research has focused on the application of sequence-based [9][10][11][12][18][19][20][21] models. Du et al [9] designed the DeepLog framework using LSTM neural networks to realize online anomaly detection on system logs.…”

Section: Related Workmentioning

confidence: 99%

“…Te idea behind Logsy is that the auxiliary dataset is sufciently informative to enhance the representation of the normal data, yet diverse enough to regularize against overftting and improve generalization. Steverson et al [19] detect attacks on an enterprise network by applying mining NLP techniques to Windows Event Logs (WELs), using transformer models and self-supervised training methods. A self-supervised anomaly detection model was constructed by combining deep learning methods, traditional machine learning, and natural language processing.…”

Section: Related Workmentioning

confidence: 99%

LogPal: A Generic Anomaly Detection Scheme of Heterogeneous Logs for Network Systems

Lei

2023

Security and Communication Networks

View full text Add to dashboard Cite

As a key resource for diagnosing and identifying problems, network syslog contains vast quantities of information. And it is the main source of data for anomaly detection of systems. Syslog presents the characteristics of large scale, diverse types and sources, data noise, and quick evolvement, which makes the detection methods not generic enough. To effectively address problem of log anomaly labelling caused by massive heterogeneous logs, we propose LogPal, a generic anomaly detection scheme of heterogeneous logs for network systems, which innovatively combines template sequences and raw log sequences to construct and generate log pattern events. By improving the self-attention mechanism of transformer, LogPal proactively synthesizes self-attention and handles log pattern events in a unique way. The model can make full use of log template and sequence semantic information, by automatically becoming aware of the pattern of logs. We implemented experiments to evaluate the performance of LogPal on publicly available datasets, and the outcome of the experiments shows that LogPal automatically adapts to log type changes and improves precision, recall, and F1 score to 99% on publicly available datasets.

show abstract

“…In Steverson et. al., 30 their CALBAC program was able to successfully train transformer models to recognize statistical differences between insample and out-of-sample Windows event log files, thus allowing for the creation of powerful file-based anomaly detection software. In this work, we use CLAPBAC, 31 a network-based anomaly detection toolkit which enables the training of models with a transformer architecture analogous to those produced with CALBAC but which train on the headers of network packet captures rather than on system log files.…”

Section: Transformer Modelingmentioning

confidence: 99%

Autonomous cyber warfare agents: dynamic reinforcement learning for defensive cyber operations

Bierbrauer¹,

Schabinger²,

Carlin³

et al. 2023

Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications V

View full text Add to dashboard Cite

In this work, we aim to develop novel cybersecurity playbooks by exploiting dynamic reinforcement learning (RL) methods to close holes in the attack surface left open by the traditional signature-based approach to Defensive Cyber Operations (DCO). A useful first proof-of-concept is provided by the problem of training a scanning defense agent using RL; as a first line of defense, it is important to protect sensitive networks from network mapping tools. To address this challenge, we developed a hierarchical, Monte Carlo-based RL framework for the training of an autonomous agent which detects and reports the presence of Nmap scans in near real-time, efficiently and with near-perfect accuracy. Our algorithm is powered by a reduction of the state space given by a transformer, CLAPBAC, an anomaly detection tool which applies natural language processing to cybersecurity in a manner consistent with state-of-the-art. In a realistic scenario emulated in CyberVAN, our approach generates optimized playbooks for effective defense against malicious insiders inappropriately probing sensitive networks.

show abstract

Cyber Intrusion Detection using Natural Language Processing on Windows Event Logs

Cited by 6 publications

References 8 publications

Redefining Malware Sandboxing: Enhancing Analysis Through Sysmon and ELK Integration

Redefining Malware Sandboxing: Enhancing Analysis Through Sysmon and ELK Integration

LogPal: A Generic Anomaly Detection Scheme of Heterogeneous Logs for Network Systems

Autonomous cyber warfare agents: dynamic reinforcement learning for defensive cyber operations

Contact Info

Product

Resources

About