Network traffic verification based on a public dataset for IDS systems and machine learning classification algorithms

Fosić, I.; Žagar, Drago; Grgić, Krešimir

doi:10.23919/mipro55190.2022.9803674

Cited by 3 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This discrimination threshold line is presented in 45-degree red diagonal line in Figure 10b,d. The AUC values closer to 1 emphasize better classification capability, whereas results at 0.5 indicate the model's poor capability to classify the data points [39]. By observing the AUC score, the improvement from the Diagnostics validation (AUC = 0.774) (Figure 10b) to the Comparative validation (AUC = 0.928) (Figure 10d) is generally comparable due to inclusion of DS2 data; thus, showing the model's above average increase in classification capability with the help of simulated (SN and SA) x6 torque data.…”

Section: Comparative-validation Resultsmentioning

confidence: 99%

Improving Deep Learning Anomaly Diagnostics with a Physics-Based Simulation Model

Mäkiaho,

Koskinen,

Laitinen

2024

Applied Sciences

View full text Add to dashboard Cite

Deep learning algorithms often struggle to accurately distinguish between healthy and anomalous states due to the scarcity of high-quality data in real-world applications. However, these data can be obtained through a physics-based simulation model. In this research, the model serves a dual purpose: detecting anomalies in industrial processes and replicating the machine’s operational behavior with high fidelity in terms of a simulated torque signal. When anomalous behaviors are detected, their patterns are utilized to generate anomalous events, contributing to the enhancement of deep neural network model training. This research proposes a method, named Simulation-Enhanced Anomaly Diagnostics (SEAD), to detect anomalies and further create high-quality data related to the diagnosed faults in the machine’s operation. The findings of this study suggest that employing a physics-based simulation model as a synthetic-anomaly signal generator can significantly improve the classification accuracy of identified anomalous states, thereby enhancing the deep learning model’s ability to recognize deviating behavior at an earlier stage when more high-quality data of the identified anomaly has been available for the learning process. This research measures the classification capability of a Long Short-Term Memory (LSTM) autoencoder to classify anomalous behavior in different SEAD stages. The validated results clearly demonstrate that simulated data can contribute to the LSTM autoencoder’s ability to classify anomalies in a peripheral milling machine. The SEAD method is employed to test its effectiveness in detecting and replicating a failure in the support element of the peripheral milling machine.

show abstract

Section: Comparative-validation Resultsmentioning

confidence: 99%

Improving Deep Learning Anomaly Diagnostics with a Physics-Based Simulation Model

Mäkiaho,

Koskinen,

Laitinen

2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Examples of instances include [17], where the F1-score remains suboptimal, indicating the model is not achieving adequate performance on the minority class, even though overall accuracy appears high. In a study employing the UNSW-NB15 dataset [11], even though the dataset is multiclass imbalanced, the primary emphasis lies on presenting overall performance rather than individual class results. The findings demonstrate that Random Forest attains the best Area Under the Curve (AUC) and F2 scores.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Therefore, a comprehensive investigation is needed to identify the high-performance algorithm that overcomes the imbalanced class distribution in the absence of sampling methods to rebalance the distribution. Additionally, the limited reporting of individual class results, as observed in [11], poses a gap in our understanding of algorithmic vulnerabilities and strengths across diverse attack types. Lastly, despite extensive algorithm testing, a systematic exploration of the suitability of different machine learning algorithm families for multiclass imbalanced datasets is lacking.…”

Section: Literature Reviewmentioning

confidence: 99%

“…The experimental evaluation utilizes the publicly available UNSW-NB15 dataset [11], characterized by a highly imbalanced class distribution. Initial experiments compared the performance of a single Decision Tree (DT) against instancebased methods like K-Nearest Neighbor (KNN), functionbased models including Multilayer Perceptron (MLP), and Bayesian-based approaches exemplified by Naive Bayes (NB).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating Tree-based Ensemble Strategies for Imbalanced Network Attack Classification

Soon,

Amir,

Nishizaki

et al. 2024

IJACSA

View full text Add to dashboard Cite

With the continual evolution of cybersecurity threats, the development of effective intrusion detection systems is increasingly crucial and challenging. This study tackles these challenges by exploring imbalanced multiclass classification, a common situation in network intrusion datasets mirroring realworld scenarios. The paper aims to empirically assess the performance of diverse classification algorithms in managing imbalanced class distributions. Experiments were conducted using the UNSW-NB15 network intrusion detection benchmark dataset, comprising ten highly imbalanced classes. The evaluation includes basic, traditional algorithms like the Decision Tree, K-Nearest Neighbor, and Gaussian Naive Bayes, as well as advanced ensemble methods such as Gradient Boosted Decision Trees (GraBoost) and AdaBoost. Our findings reveal that the Decision Tree surpassed the Multi-Layer Perceptron, K-Nearest Neighbor, and Naive Bayes in terms of overall F1-score. Furthermore, thorough evaluations of nine tree-based ensemble algorithms were performed, showcasing their varying efficacy. Bagging, Random Forest, ExtraTrees, and XGBoost achieved the highest F1-scores. However, in individual class analysis, XGBoost demonstrated exceptional performance relative to the other algorithms. This is confirmed by achieving the highest F1-scores in eight out of the ten classes within the dataset. These results establish XGBoost as a predominant method for handling multiclass imbalance classification with Bagging being the closest feasible alternative, as Bagging gains an almost similar accuracy and F1-score as XGBoost.

show abstract

Classifying Relevant Causes of Employee Absenteeism Using Machine Learning

Fosić

Živković

2023

2023 46th MIPRO ICT and Electronics Convention (MIPRO)

View full text Add to dashboard Cite

Network traffic verification based on a public dataset for IDS systems and machine learning classification algorithms

Cited by 3 publications

References 14 publications

Improving Deep Learning Anomaly Diagnostics with a Physics-Based Simulation Model

Improving Deep Learning Anomaly Diagnostics with a Physics-Based Simulation Model

Evaluating Tree-based Ensemble Strategies for Imbalanced Network Attack Classification

Classifying Relevant Causes of Employee Absenteeism Using Machine Learning

Contact Info

Product

Resources

About