Overcoming the Challenges of Data Lack, Leakage, and Dimensionality in Intrusion Detection Systems: A Comprehensive Review

,; Bouke, Mohamed; Abdullah, Azizol; Udzir, Nur; Samian, Normalia

doi:10.14209/jcis.2024.3

Cited by 3 publications

(1 citation statement)

References 52 publications

(73 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Temporal leakage is a frequent issue in time-series data, occurring when models are trained using future data. Lastly, data preprocessing leakage may happen during normalization and encoding steps if they involve information from the full dataset [10]- [12].…”

Section: Introductionmentioning

confidence: 99%

Implications of Data Leakage in Machine Learning Preprocessing: A Multi-Domain Investigation

Bouke,

Zaid,

Abdullah

2024

Preprint

View full text Add to dashboard Cite

Data leakage during machine learning (ML) preprocessing is a critical issue where unintended external information skews the training process, resulting in artificially high-performance metrics and undermining model reliability. This study addresses the insufficient exploration of data leakage across diverse ML domains, highlighting the necessity of comprehensive investigations to ensure robust and dependable ML models in real-world applications. Significant discrepancies in model performance due to data leakage were observed, with notable variations in F1 scores and ROC AUC values for the Breast Cancer dataset. The Tic-Tac-Toe Endgame dataset analysis revealed the varying impact on models like Ridge, SGD, GaussianNB, and MLP, underscoring the profound effect of data leakage. The German Credit Scoring dataset showed slight enhancements in recall and F1 scores for models like DT and GB without data leakage, indicating reduced overfitting. Additionally, models such as PassiveAggressive, Ridge, SGD, GaussianNB, and Nearest Centroid exhibited shifts in performance metrics, highlighting the intricate response to data leakage. The study also revealed raw data leakage rates, such as 6.79% for Spambase and 1.99% for Breast Cancer. These findings emphasize meticulous data management and validation to mitigate leakage effects, which is crucial for developing reliable ML models.

show abstract

Section: Introductionmentioning

confidence: 99%

Implications of Data Leakage in Machine Learning Preprocessing: A Multi-Domain Investigation

Bouke,

Zaid,

Abdullah

2024

Preprint

View full text Add to dashboard Cite

show abstract

Towards robust and efficient intrusion detection in IoMT: a deep learning approach addressing data leakage and enhancing model generalizability

Bouke,

El Atigh,

Abdullah

2024

Multimed Tools Appl

View full text Add to dashboard Cite

Application of BukaGini algorithm for enhanced feature interaction analysis in intrusion detection systems

Bouke,

Abdullah,

Cengiz

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

This article presents an evaluation of BukaGini, a stability-aware Gini index feature selection algorithm designed to enhance model performance in machine learning applications. Specifically, the study focuses on assessing BukaGini’s effectiveness within the domain of intrusion detection systems (IDS). Recognizing the need for improved feature interaction analysis methodologies in IDS, this research aims to investigate the performance of BukaGini in this context. BukaGini’s performance is evaluated across four diverse datasets commonly used in IDS research: NSLKDD (22,544 samples), WUSTL EHMS (16,318 samples), WSN-DS (374,661 samples), and UNSWNB15 (175,341 samples), amounting to a total of 588,864 data samples. The evaluation encompasses key metrics such as stability score, accuracy, F1-score, recall, precision, and ROC AUC. Results indicate significant advancements in IDS performance, with BukaGini achieving remarkable accuracy rates of up to 99% and stability scores consistently surpassing 99% across all datasets. Additionally, BukaGini demonstrates an average reduction in dimensionality of 25%, selecting 10 features for each dataset using the Gini index. Through rigorous comparative analysis with existing methodologies, BukaGini emerges as a promising solution for feature interaction analysis within cybersecurity applications, particularly in the context of IDS. These findings highlight the potential of BukaGini to contribute to robust model performance and propel intrusion detection capabilities to new heights in real-world scenarios.

show abstract

Overcoming the Challenges of Data Lack, Leakage, and Dimensionality in Intrusion Detection Systems: A Comprehensive Review

Cited by 3 publications

References 52 publications

Implications of Data Leakage in Machine Learning Preprocessing: A Multi-Domain Investigation

Implications of Data Leakage in Machine Learning Preprocessing: A Multi-Domain Investigation

Towards robust and efficient intrusion detection in IoMT: a deep learning approach addressing data leakage and enhancing model generalizability

Application of BukaGini algorithm for enhanced feature interaction analysis in intrusion detection systems

Contact Info

Product

Resources

About