Similarity as a central approach to flow‐based anomaly detection

Abstract

SUMMARY Network flow monitoring is currently a common practice in mid‐ and large‐size networks. Methods of flow‐based anomaly detection are subject to ongoing extensive research, because detection methods based on deep packets have reached their limits. However, there is a lack of comprehensive studies mapping the state of the art in this area. For this reason, we have conducted a thorough survey of flow‐based anomaly detection methods published on academic conferences and used by the industry. We have analyzed these methods using the perspective of similarity which is inherent to any anomaly detection method. Based on this analysis, we have proposed a new taxonomy of network anomalies and a similarity‐oriented classification of flow‐based detection methods. We have also identified four issues requiring further research: the lack of flow‐based evaluation datasets, infeasible benchmarking of proposed methods, excessive false positive rate and limited coverage of certain anomaly classes. Copyright © 2014 John Wiley & Sons, Ltd.

Botnet detection based on network flow summary and deep learning

“…The experimental results show that flow‐based features, such as interarrival time features, are the most representative features to model botnet communication and the highest classification accuracy is reached by C4.5 algorithm. The feature selection to detect anomaly was elaborated by Drasar et al The impact of flow‐based features to detection accuracy enhancement is evaluated.…”

Novel intrusion prediction mechanism based on honeypot log similarity

“…Because of the increasing data flow rate of the Internet, the system performance of signature-based detection is falling. Consequently, some studies [21][22][23][24][25][26][27][28][29] proposed a new flow-based detection mechanism, which combines signature-based and anomaly-based methodologies with correlated honeypot logs. Of these studies, only [27,28] have prediction capability.…”

“…In [39], the authors utilized association rule mining using a modified Apriori algorithm to distinguish malicious flows from suspicious flows collected by histogram-based detectors [40]. It uses Mahalanobis distance for anomaly detection by means of [21] Signature based and anomaly based No Thakar et al [18] Content based No Dressler et al [22] Signature based and anomaly based No Wei-wei et al [30] Anomaly based Yes Zhan et al [31] Signature based Yes Artail et al [23] Signature based and anomaly based No Drašar et al [24] Signature based and anomaly based No Curran et al [3] Signature based No Kwon et al [32] Signature based Yes Tudorica et al [17] Signature based No Singh et al [25] Signature based and anomaly based No Jain et al [26] Signature based and anomaly based No Alosefer et al [27] Signature based and anomaly based Yes Ma et al [28] Signature based and anomaly based Yes Zhao et al [29] Signature based and anomaly based No clustering and further uses association rule mining to reduce the false positive rates. But the major drawback of Mahalanobis distance is that it requires the inversion of the covariance matrix, which may result in incorrect values.…”

Adaptive traffic sampling for P2P botnet detection

“…Most current P2P botnet detection systems examine network traffic to detect active bots. () However, with the advent of multi‐gigabit link speeds, capturing and analyzing header and payload of every packet requires enormous amounts of computational resources and is therefore not feasible in high‐speed and high‐volume networks. To solve this problem, sampling techniques are widely used in these situations to allow the analysis of high traffic volumes with limited resources.…”