Direct Batch Growth Hierarchical Self-Organizing Mapping Based on Statistics for Efficient Network Intrusion Detection

Abstract

A new evaluation mechanism was proposed to enhance the representation of data topology in the directed batch growth hierarchical self-organizing mapping. In the proposed mechanism, the growth threshold and the correlation worked in a case-sensitive manner through the statistic calculation of the input data. Since the proposed model enabled a more thorough representation of data topology from both the horizontal and the vertical directions, it naturally held great potential in detecting various traffic attacks. Numerical experiments of network intrusion detection were carried out on the datasets of KDD99, Moore and CICIDS2017, where the good performance validated the superiority of the proposed method.INDEX TERMS Network intrusion detection, growth hierarchical self-organizing mapping, statistical enhancement, growth threshold, correlation.

Entropy-Defined Direct Batch Growing Hierarchical Self-Organizing Mapping for Efficient Network Anomaly Detection

“…In 2020, we proposed an SE-DBGHSOM [8], which is an improvement of dynamic SOM to network traffic anomaly detection. Both SE-DBGHSOM and the ED-DBGHSOM are improvements for the dynamic SOM (GSOM, GHSOM).…”

“…Therefore, both SE-DBGHSOM and ED-DBGHSOM have newly defined neuron growth thresholds. SE-DBGHSOM defines the neuron growth threshold based on statistical standard deviation [8], while ED-DBGHSOM defines the neuron growth threshold based on information theory entropy. The purpose of both SE-DBGHSOM and ED-DBGHSOM is to identify attack traffic by using SOM to accurately map the topology of network traffic.…”

“…The mean effect will lead to lower attack detection rate and higher false positive rate. [5] 99.63% 1.8% SVM [27] 98.97% 4.02% Eskin SVM [28] 98.00% 10.00% Yu [29] 96.02% 4.92% Sarasamma [30] 93.46% 3.99% K-NN [31] 93.39% 28.69% Eskin Clustering [28] 93.00% 10.00% SE-DBGSOM [7] 91.27% 31.74% GHSOM [7] 90.87% 2.69% LDA-SVM [28] 89.93% 2.03% DBGSOM [6] 89.45% 14.19% Sequential classifiers [27] 71.19% 27.42% Set2 ED-DBGHSOM(with REP ) 97.22% 1.35% ED-DBGHSOM 86.11% 1.41% ED-DBGSOM 58.33% 3.61% SE-DBGHSOM [8] 13.89% 1.17% SE-DBGSOM [7] 11.11% 1.1% GHSOM 0% 0% Set3 ED-DBGHSOM 100% 0.09% ED-DBGSOM 99.98% 0.06% DBGSOM [6] 97.55% 0.32% SE-DBGSOM [7] 99.85% 0.65% GHSOM [7] 99.72% 5.92% reversible sketch [ Therefore, this paper uses entropy to define SOM architecture, abandoning the traditional SOM architecture based on cumulative error, and providing a new direction for the development of dynamic SOM. CDF is introduced in SE-DBGHSOM to cluster the data which cannot be classified by SE-DBGHSOM.…”

“…To overcome the threat of increasing network intrusion, a number of methods with respect to the anomaly network traffic detection have been developed, including principal component analysis, wavelet analysis, Markov model, clustering, histogram, sketch [1] and neural network [2]. Therefore, the arguably most efficient method is clustering, where over 100 anomaly detection studies based on self-organizing mapping (SOM) have been carried out since 2000 [3], including G-SOM [4], GHSOM [5], DBGSOM [6], SE-DBGSOM [7], SE-DBGHSOM [8]. However, these models use either the quantization error or the Euclidean distance of data vectors to determine neuron growth, which may come across the challenge of inaccurate data clustering.…”