Generating Labeled Flow Data from MAWILab Traces for Network Intrusion Detection

Kim, Jin Oh; Sim, Caitlin; Choi, Jinhwan

doi:10.1145/3322798.3329251

Cited by 10 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The MAWILab logs contain the attack information collected from multiple intrusion detections systems against the captured packets. In our previous work [38], we combined the captured packet traces with the IDS logs to generate the labeled data for network anomaly detection. A single day trace contains the packets captured in the 15-minute interval, and the number of data points for a day is tens of millions.…”

Section: ) Mawilab Dataset [30]mentioning

confidence: 99%

An Empirical Evaluation of Deep Learning for Network Anomaly Detection

et al. 2019

Self Cite

View full text Add to dashboard Cite

Deep learning has been widely studied in many technical domains such as image analysis and speech recognition, with its benefits that effectively deal with complex and high-dimensional data.Our preliminary experiments show a high degree of non-linearity from the network connection data, which explains why it is hard to improve the performance of identifying network anomalies by using conventional learning methods (e.g., Adaboosting, SVM, and Random Forest). In this study, we design and examine deep learning models constructed based on Fully Connected Networks (FCNs), Variational AutoEncoder (VAE), and Sequence-to-Sequence (Seq2Seq) structures. For the extensive evaluation, we employ a broad range of the public datasets with unique characteristics. Our experimental results confirm the feasibility of deep learning-based network anomaly detection, with the improved performance compared to the conventional learning techniques. In particular, the detection model based on Seq2Seq with LSTM is highly promising, consistently yielding over 99% of accuracy to identify network anomalies from the entire datasets employed in the evaluation.

show abstract

Section: ) Mawilab Dataset [30]mentioning

confidence: 99%

An Empirical Evaluation of Deep Learning for Network Anomaly Detection

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, new solutions to public datasets limitations had been proposed. As an example, new software tools had emerged, such as Silk [19]. The authors proposed a method to produce network traffic datasets for NIDS research by extracting meta-information from the network packets, combined with the logs from a IDS to label the traffic.…”

Section: Related Workmentioning

confidence: 99%

A Novel Framework for Generating Personalized Network Datasets for NIDS Based on Traffic Aggregation

Alvarado

Gonzalez

Martínez-Peláez

et al. 2022

Sensors

View full text Add to dashboard Cite

In this paper, we addressed the problem of dataset scarcity for the task of network intrusion detection. Our main contribution was to develop a framework that provides a complete process for generating network traffic datasets based on the aggregation of real network traces. In addition, we proposed a set of tools for attribute extraction and labeling of traffic sessions. A new dataset with botnet network traffic was generated by the framework to assess our proposed method with machine learning algorithms suitable for unbalanced data. The performance of the classifiers was evaluated in terms of macro-averages of F1-score (0.97) and the Matthews Correlation Coefficient (0.94), showing a good overall performance average.

show abstract

“…Each pcap file is a recording of 15-minute traffic collected on a specific day. A previous study in [15] constructed network data by combining these traffic traces and intrusion logs, but the resulting dataset contains NetFlow-like statistical information made only available at the connection release time. In this work, we construct packet stream data to develop the function for the early identification of network attacks.…”

Section: Packet Stream Data Constructionmentioning

confidence: 99%

Deep Sequence Models for Packet Stream Analysis and Early Decisions

Kim

Lee

et al. 2022

2022 IEEE 47th Conference on Local Computer Networks (LCN)

Self Cite

View full text Add to dashboard Cite

The packet stream analysis is essential for the early identification of attack connections while in progress, enabling timely responses to protect system resources. However, there are several challenges for implementing effective analysis, including out-of-order packet sequences introduced due to network dynamics and class imbalance with a small fraction of attack connections available to characterize. To overcome these challenges, we present two deep sequence models: (i) a bidirectional recurrent structure designed for resilience to out-of-order packets, and (ii) a pre-training-enabled sequence-to-sequence structure designed for better dealing with unbalanced class distributions using selfsupervised learning. We evaluate the presented models using a real network dataset created from month-long real traffic traces collected from backbone links with the associated intrusion log. The experimental results support the feasibility of the presented models with up to 94.8% in F1 score with the first five packets (k=5), outperforming baseline deep learning models.

show abstract

Generating Labeled Flow Data from MAWILab Traces for Network Intrusion Detection

Cited by 10 publications

References 11 publications

An Empirical Evaluation of Deep Learning for Network Anomaly Detection

An Empirical Evaluation of Deep Learning for Network Anomaly Detection

A Novel Framework for Generating Personalized Network Datasets for NIDS Based on Traffic Aggregation

Deep Sequence Models for Packet Stream Analysis and Early Decisions

Contact Info

Product

Resources

About