An improved X-means and isolation forest based methodology for network traffic anomaly detection

Feng, Yifan; Cai, Wei; Yue, Haoyu; Lin, Yan; Chen, Jiaxin; Hu, Zhiqiang

doi:10.1371/journal.pone.0263423

Cited by 10 publications

(3 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ensemble learning is usually a combination of a machine learning approach with another machine learning approach of the same category (e.g., supervised with supervised) for improving the accuracy and robustness of the predictions. An anomaly detection algorithm by combining X-means and isolation forest ML models using ensemble learning has shown better classification performance compared to the other unsupervised machine learning approaches' classification performance considered individually [177]. An ensemble learning approach has been proposed by combining K-nearest neighbor, Naive Bayes, support vector machines, and Self-Organizing Maps (SOMs) to detect anomalous behavior of data traffic in KDN, where the combination of SVM and SOM has resulted in high accuracy and detection rates with a low false alarm rate [178].…”

Section: Generating Knowledge Using Machine Learning Methodsmentioning

confidence: 99%

A Comprehensive Survey on Knowledge-Defined Networking

Wijesekara

Gunawardena

2023

Telecom

View full text Add to dashboard Cite

Traditional networking is hardware-based, having the control plane coupled with the data plane. Software-Defined Networking (SDN), which has a logically centralized control plane, has been introduced to increase the programmability and flexibility of networks. Knowledge-Defined Networking (KDN) is an advanced version of SDN that takes one step forward by decoupling the management plane from control logic and introducing a new plane, called a knowledge plane, decoupled from control logic for generating knowledge based on data collected from the network. KDN is the next-generation architecture for self-learning, self-organizing, and self-evolving networks with high automation and intelligence. Even though KDN was introduced about two decades ago, it had not gained much attention among researchers until recently. The reasons for delayed recognition could be due to the technology gap and difficulty in direct transformation from traditional networks to KDN. Communication networks around the globe have already begun to transform from SDNs into KDNs. Machine learning models are typically used to generate knowledge using the data collected from network devices and sensors, where the generated knowledge may be further composed to create knowledge ontologies that can be used in generating rules, where rules and/or knowledge can be provided to the control, management, and application planes for use in decision-making processes, for network monitoring and configuration, and for dynamic adjustment of network policies, respectively. Among the numerous advantages that KDN brings compared to SDN, enhanced automation and intelligence, higher flexibility, and improved security stand tall. However, KDN also has a set of challenges, such as reliance on large quantities of high-quality data, difficulty in integration with legacy networks, the high cost of upgrading to KDN, etc. In this survey, we first present an overview of the KDN architecture and then discuss each plane of the KDN in detail, such as sub-planes and interfaces, functions of each plane, existing standards and protocols, different models of the planes, etc., with respect to examples from the existing literature. Existing works are qualitatively reviewed and assessed by grouping them into categories and assessing the individual performance of the literature where possible. We further compare and contrast traditional networks and SDN against KDN. Finally, we discuss the benefits, challenges, design guidelines, and ongoing research of KDNs. Design guidelines and recommendations are provided so that identified challenges can be mitigated. Therefore, this survey is a comprehensive review of architecture, operation, applications, and existing works of knowledge-defined networks.

show abstract

Section: Generating Knowledge Using Machine Learning Methodsmentioning

confidence: 99%

A Comprehensive Survey on Knowledge-Defined Networking

Wijesekara

Gunawardena

2023

Telecom

View full text Add to dashboard Cite

show abstract

“…Security-preserving data collecting and sharing scheme Anomaly detection and fault discrimination have been investigated in many existing works [60,61]. Nevertheless, with the advent of the fifth generation (5G), the amount of data has significantly increased, which causes potential data anomalies and operating faults [62,63]. Specifically, challenges for anomaly detection and fault discrimination methods arise, as illustrated in Fig.…”

Section: Data Acquisition System Based On Blockchainmentioning

confidence: 99%

Intelligent Identification over Power Big Data: Opportunities, Solutions, and Challenges

Luo¹,

Li²,

Yang³

et al. 2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

The emergence of power dispatching automation systems has greatly improved the efficiency of power industry operations and promoted the rapid development of the power industry. However, with the convergence and increase in power data flow, the data dispatching network and the main station dispatching automation system have encountered substantial pressure. Therefore, the method of online data resolution and rapid problem identification of dispatching automation systems has been widely investigated. In this paper, we perform a comprehensive review of automated dispatching of massive dispatching data from the perspective of intelligent identification, discuss unresolved research issues and outline future directions in this area. In particular, we divide intelligent identification over power big data into data acquisition and storage processes, anomaly detection and fault discrimination processes, and fault tracing for dispatching operations during communication. A detailed survey of the solutions to the challenges in intelligent identification over power big data is then presented. Moreover, opportunities and future directions are outlined.

show abstract

“…Time series anomaly detection is an important problem with applications in various domains such as manufacturing, medical, and engineering [1][2][3][4][5][6][7][8][9]. Generally, an anomaly changing with time [10] can be a point anomaly (i.e., a single value beyond a regular range) or a sequence anomaly (i.e., an abnormal waveform denoted by a sequence of data points) [11,12].…”

Section: Introductionmentioning

confidence: 99%

GraphTS: Graph-represented time series for subsequence anomaly detection

Zarei

Huang

2023

PLoS ONE

View full text Add to dashboard Cite

Automatic detection of subsequence anomalies (i.e., an abnormal waveform denoted by a sequence of data points) in time series is critical in a wide variety of domains. However, most existing methods for subsequence anomaly detection often require knowing the length and the total number of anomalies in time series. Some methods fail to capture recurrent subsequence anomalies due to using only local or neighborhood information for anomaly detection. To address these limitations, in this paper, we propose a novel graph-represented time series (GraphTS) method for discovering subsequence anomalies. In GraphTS, we provide a new concept of time series graph representation model, which represents both recurrent and rare patterns in a time series. Particularly, in GraphTS, we develop a new 2D time series visualization (2Dviz) method, which compacts all 1D time series patterns into a 2D spatial temporal space. The 2Dviz method transfers time series patterns into a higher-resolution plot for easier sequence anomaly recognition (or detecting subsequence anomalies). Then, a Graph is constructed based on the 2D spatial temporal space of time series to capture recurrent and rare subsequence patterns effectively. The represented Graph also can be used to discover single and recurrent subsequence anomalies with arbitrary lengths. Experimental results demonstrate that the proposed method outperforms the state-of-the-art methods in terms of accuracy and efficiency.

show abstract

An improved X-means and isolation forest based methodology for network traffic anomaly detection

Cited by 10 publications

References 38 publications

A Comprehensive Survey on Knowledge-Defined Networking

A Comprehensive Survey on Knowledge-Defined Networking

Intelligent Identification over Power Big Data: Opportunities, Solutions, and Challenges

GraphTS: Graph-represented time series for subsequence anomaly detection

Contact Info

Product

Resources

About