Dimensionality Reduction and Visualization of Network Intrusion Detection Data

Zong, Wei; Chow, Yang-Wai; Susilo, Willy

doi:10.1007/978-3-030-21548-4_24

Cited by 7 publications

(6 citation statements)

References 19 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The direction, inter-packet length, and inter-arrival times are the most important properties in the flow-based feature formulation: total duration (dur) and destination-to-source-time-to-live (dttl) are two examples of flow-based features. The features are categorized into three sets, namely basic (6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18), content (19)(20)(21)(22)(23)(24)(25)(26), and time (27)(28)(29)(30)(31)(32)(33)(34)(35). Features 36-40 and 41-47 are labeled as general-purpose features and connection features, respectively.…”

Section: Unsw-nb15 Datasetmentioning

confidence: 99%

“…The UNSW-NB15 is one of the popular [14][15][16][17] and comprehensive cybersecurity datasets released in 2015. 18,19 This dataset is comprised of 2 540 044 realistic modern normal and abnormal (also known as an attack) network activities.…”

Section: Unsw-nb15 Datasetmentioning

confidence: 99%

“…Karami 14 proposed a modified Self‐Organizing Map (SOM) to both detect the anomalies and provide the information summary from 2D data visualization using UNSW‐NB15. Similarly, References 15‐17 visualized the UNSW‐NB15 data points. They utilized PCA to reduce the dimensionality and keep the first three principal components.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

UNSW‐NB15 computer security dataset: Analysis through visualization

Zoghi

Serpen

2023

Security and Privacy

View full text Add to dashboard Cite

Class imbalance refers to a major issue in data mining where data with unequal class distribution can deteriorate classification performance. Although it alone affects the performance of the classifiers, the joint‐effect of class imbalance and overlap is more damaging. Data overlap happens when multiple classes are assigned to a single data point causing the classifiers to misidentify the class boundaries. This study offers a deep insight into the intricacies of the UNSW‐NB15 dataset and two issues that may lead the data‐driven models to demonstrate poor performance. The most commonly used visualization methods such as bar chart, 3D and 2D scatter plots, intercluster distance map, and parallel coordinate diagram were employed to depict the data imbalanced and overlap. However, their limitations in capturing the overlapping issue led us to propose an accurate, easy‐to‐interpret, and scalable overlapping visualization method. The method clearly detects the data overlap and illustrates the effect of several data scalers in dealing with the data overlap. To verify the accuracy of the proposed method, a number of classifiers were implemented along with the scalers and the calculated AUC scores were compared to those calculated from the classifiers that were implemented on the original dataset.

show abstract

Section: Unsw-nb15 Datasetmentioning

confidence: 99%

Section: Unsw-nb15 Datasetmentioning

confidence: 99%

See 1 more Smart Citation

UNSW‐NB15 computer security dataset: Analysis through visualization

Zoghi

Serpen

2023

Security and Privacy

View full text Add to dashboard Cite

show abstract

“…Ideally, we would like to extract the features automatically for different datasets. The two most widely used automatic feature extraction methods are principal component analysis (PCA) and autoencoder (AE) [5] . Furthermore, the training process for both methods does not require labels, which solves the problem of manual labeling caused by the huge network traffic.…”

Section: Introductionmentioning

confidence: 99%

Research on Dung Beetle Optimization Based Stacked Sparse Autoencoder for Network Situation Element Extraction

Yang,

Zhao

2024

IEEE Access

View full text Add to dashboard Cite

Network security situation awareness enables networks to actively and effectively defend against network attacks, relying on the extraction of network situation elements as an initial and decisive step. In existing studies, the stacked sparse autoencoder (SSAE) has been employed to extract features from unlabeled network flows. However, obtaining the optimal hyperparameter combination is challenging due to its numerous hyperparameters. To address this issue, we propose a novel approach named DBO-SSAE that leverages dung beetle optimization (DBO) to select the optimal hyperparameters for SSAE automatically. Applied to the well-known UNSW-NB15 dataset, our model yields an optimal feature subset, which is evaluated across various binary classifiers with different metrics. Experimental results demonstrate that our approach improves accuracy and F1-measure by 0.2% to 1.5%, while reducing the false negative rate (FNR) and false positive rate (FPR) by 0.06% to 7%, surpassing other feature extraction methods on the same classifier for the UNSW-NB15 dataset. Particularly, in conjunction with a lightweight bidirectional long short-term memory (BiLSTM), our model achieves metrics of 98.84% accuracy, 98.96% F1-measure, 1.86% FNR, and 0.6% FPR. This study could provide novel insights into the effective representation of network situation elements and lay the groundwork for a high-efficiency intrusion detection system.

show abstract

“…With the development of capable AI-driven IDS/IPS technologies, there have been various studies investigating and developing data-driven methods. Besides the data analytics and pattern presentations using traditional visualization techniques [3] and unsupervised K-means clustering [4] , the method toward detecting attacks can be mainly divided into two parts, the classical machine learning classifiers and deep learning models. In classical learning methods, several classifiers are utilized and modified for binary and multi-class intrusion detection tasks [5]- [8], including basic tree methods, Multilayer Perceptron, and Support Vector Machine, Naive Bayes, Random Forest, and a sophisticated variant of boost-based classifiers.…”

Section: Introductionmentioning

confidence: 99%

Learning to Detect: A Data-driven Approach for Network Intrusion Detection

Zachary¹,

Jiang²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

With massive data being generated daily and the ever-increasing interconnectivity of the world's Internet infrastructures, a machine learning based intrusion detection system (IDS) has become a vital component to protect our economic and national security. In this paper, we perform a comprehensive study on NSL-KDD, a network traffic dataset, by visualizing patterns and employing different learning-based models to detect cyber attacks. Unlike previous shallow learning and deep learning models that use the single learning model approach for intrusion detection, we adopt a hierarchy strategy, in which the intrusion and normal behavior are classified firstly, and then the specific types of attacks are classified. We demonstrate the advantage of the unsupervised representation learning model in binary intrusion detection tasks. Besides, we alleviate the data imbalance problem with SVM-SMOTE oversampling technique in 4-class classification and further demonstrate the effectiveness and the drawback of the oversampling mechanism with a deep neural network as a base model.

show abstract

Dimensionality Reduction and Visualization of Network Intrusion Detection Data

Cited by 7 publications

References 19 publications

UNSW‐NB15 computer security dataset: Analysis through visualization

UNSW‐NB15 computer security dataset: Analysis through visualization

Research on Dung Beetle Optimization Based Stacked Sparse Autoencoder for Network Situation Element Extraction

Learning to Detect: A Data-driven Approach for Network Intrusion Detection

Contact Info

Product

Resources

About