Resampling imbalanced data for network intrusion detection datasets

Abstract

Machine learning plays an increasingly significant role in the building of Network Intrusion Detection Systems. However, machine learning models trained with imbalanced cybersecurity data cannot recognize minority data, hence attacks, effectively. One way to address this issue is to use resampling, which adjusts the ratio between the different classes, making the data more balanced. This research looks at resampling’s influence on the performance of Artificial Neural Network multi-class classifiers. The resampling methods, random undersampling, random oversampling, random undersampling and random oversampling, random undersampling with Synthetic Minority Oversampling Technique, and random undersampling with Adaptive Synthetic Sampling Method were used on benchmark Cybersecurity datasets, KDD99, UNSW-NB15, UNSW-NB17 and UNSW-NB18. Macro precision, macro recall, macro F1-score were used to evaluate the results. The patterns found were: First, oversampling increases the training time and undersampling decreases the training time; second, if the data is extremely imbalanced, both oversampling and undersampling increase recall significantly; third, if the data is not extremely imbalanced, resampling will not have much of an impact; fourth, with resampling, mostly oversampling, more of the minority data (attacks) were detected.

Network intrusion detection using oversampling technique and machine learning algorithms

“…While the model proposed by Koroniotis et al (2017) achieved 93.23% accuracy using DT classifier. In addition, none of the studies listed in Table 1 have resolved the class imbalance problem of the UNSW-NB15 dataset as there are many studies ( Al-Daweri et al, 2020 ; Ahmad et al, 2021 ; Bagui & Li, 2021 ; Dlamini & Fahim, 2021 ) that have highlighted this issue. We addressed the class imbalance problem by applying SMOTE that improved the performance of the classifiers and achieved good results.…”

Developing an Efficient Feature Engineering and Machine Learning Model for Detecting IoT-Botnet Cyber Attacks

“…Pahl et al [15] describe in their paper that even though there is some related work found in IoT, still it is attracting the attention of researchers for its popularity in today's livelihood and designed an anomaly-based detector and firewall for IoT system using K-Means and BIRCH clustering with a predictive accuracy of 96.3%. Brun et al [16] Bagui and Li [26] presented the usefulness of random oversampling and random under-sampling with SMOTE to deal with highly imbalanced and less imbalanced intrusion detection datasets to improve the classification accuracy of the classifier. They evaluated the model by using artificial neural network (ANN) for attack detection using macro precision; macro recall, and macro F1 score for several sampling techniques and found that SMOTE-Random under-sampling with ANN classifier outperforms all others with 83.61% macro precision; 87.14% macro recall; 82.75% macro F1-score with 342 seconds training time.…”

Memory-Efficient Deep Learning for Botnet Attack Detection in IoT Networks

“…Liaqat et al [41] used the up-sampling method to increase the number of benign samples in the training data set. In [42][43][44][45], Synthetic Minority Oversampling Technique (SMOTE) method was used to generate additional samples for the minority classes. Mulyanto et al [46] performed feature selection to reduce dimensionality while focal loss function was used to address class imbalance problem.…”

“…Recent studies recommended SMOTE as an efficient over-sampling method [42][43][44][45]47,51]. Therefore, SMOTE algorithm was proposed to deal with the high class imbalance problem in the training set in an 11-class classification scenario.…”