On the Evaluation of Sequential Machine Learning for Network Intrusion Detection

Abstract

Recent advances in deep learning renewed the research interests in machine learning for Network Intrusion Detection Systems (NIDS). Specifically, attention has been given to sequential learning models, due to their ability to extract the temporal characteristics of Network traffic Flows (NetFlows), and use them for NIDS tasks. However, the applications of these sequential models often consist of transferring and adapting methodologies directly from other fields, without an in-depth investigation on how to leverage the specific circumstances of cybersecurity scenarios; moreover, there is a lack of comprehensive studies on sequential models that rely on NetFlow data, which presents significant advantages over traditional full packet captures. We tackle this problem in this paper. We propose a detailed methodology to extract temporal sequences of NetFlows that denote patterns of malicious activities. Then, we apply this methodology to compare the efficacy of sequential learning models against traditional static learning models. In particular, we perform a fair comparison of a 'sequential' Long Short-Term Memory (LSTM) against a 'static' Feedforward Neural Networks (FNN) in distinct environments represented by two well-known datasets for NIDS: the CICIDS2017 and the CTU13. Our results highlight that LSTM achieves comparable performance to FNN in the CICIDS2017 with over 99.5% F1-score; while obtaining superior performance in the CTU13, with 95.7% F1-score against 91.5%. This paper thus paves the way to future applications of sequential learning models for NIDS. CCS CONCEPTS• Security and privacy → Intrusion detection systems; Network security; • Computing methodologies → Temporal reasoning.

A Quantitative Logarithmic Transformation-Based Intrusion Detection System

“…Compared to existing IDS methods [3][4][5][6][12][13][14][15][16][17][18][19][20][21][22][23], the QLT-IDS system proposed in the present study has three major advantages.…”

“…Intrusion detection systems (IDS) are powerful security information and event management (SIEM) tools that help network operators protect vulnerable devices by monitoring malicious activities and automatically recording intrusion events within the network. Current IDS systems utilize three main methods to detect network attacks, namely (1) predefined rules or virus signatures [3][4][5][6]12], (2) anomaly detection [12][13][14][15][16], and (3) artificial intelligence [17][18][19][20][21][22][23][24][25][26] (machine-learning or deep-learning models). However, predefined detection methods require the identification and implementation of effective rules in advance and/or the extensive collection and update of potential virus signatures.…”

Consolidating Packet-Level Features for Effective Network Intrusion Detection: A Novel Session-Level Approach

“…Next, we surveyed related studies of ML-based NIDS that analyze session-level data. There are three cases of analyzing session-level data: when the public dataset is session-level data in the first place [31], [32], as in KDD99 and NSL-KDD; when NetFlow data is obtained from PCAP data [34], [35]; and when defining sessions originally [33], [36].…”

The Role of Machine Learning in Cybersecurity

“…In particular, we highlight those solutions that combine deep learning with temporal analyses: such twofold perspective allows to detect additional malicious patterns that can improve detection performance. For instance, in [56] the F1-score improves from 0.90 to 0.95 when also temporal dependencies are considered. We will present a real deployment of a similar solution in §7.2, describing how S2Grupo protects Industrial Control Systems (ICS), showcasing the pros (and cons) of ML with respect to older techniques based on heuristics.…”

“…Finally, an intriguing future development of such ML solution involves the consideration of 'stateful' analyses that take into account the time-axis (as done, e.g., in [56]) and allow to detect even anomalies occurring in the temporal domain. The next case-study by S2Grupo will consider a similar application.…”