Hierarchical Clustering Based Network Traffic Data Reduction for Improving Suspicious Flow Detection

Su, Liya; Yao, Yepeng; Li, Ning; Liu, Junrong; Lü, Zhigang; Liu, Baoxu

doi:10.1109/trustcom/bigdatase.2018.00108

Cited by 24 publications

(22 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In fact, some researchers are dedicated to improving the detection efficiency of network traffic. Liya et al [8] used a hierarchical clustering algorithm to divide the samples into multiple clusters. Several representative flows are selected in each cluster.…”

Section: Plos Onementioning

confidence: 99%

“…Facing this sophisticated and untrusted communication environment, this paper proposes a two-layer detection framework with a rapid rate and high precision based on the supervised learning algorithm. Current studies focus on either improving the detection accuracy [5][6][7] or optimizing the detection efficiency [8,9]. Few studies discuss how to improve the detection efficiency for a two-layer detection framework without affecting the detection accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Two-layer detection framework with a high accuracy and efficiency for a malware family over the TLS protocol

Zheng

Liu

et al. 2020

PLoS ONE

View full text Add to dashboard Cite

The transport layer security (TLS) protocol is widely adopted by apps as well as malware. With the geometric growth of TLS traffic, accurate and efficient detection of malicious TLS flows is becoming an imperative. However, current studies focus on either detection accuracy or detection efficiency, and few studies take into account both indicators. In this paper, we propose a two-layer detection framework composed of a filtering model (FM) and a malware family classification model (MFCM). In the first layer, a new set of TLS handshake features is presented to train the FM, which is devised to filter out a majority of benign TLS flows. For identifying malware families, both TLS handshake features and statistical features are applied to construct the MFCM in the second layer. Comprehensive experiments are conducted to substantiate the high accuracy and efficiency of the proposed two-layer framework. A total of 96.32% of benign TLS flows can be filtered out by the FM with few malicious TLS flows being discarded provided the threshold of the FM is set to 0.01. Moreover, a multiclassifier is selected to construct the MFCM to provide better performance than a set of binary classifiers under the same feature set. In addition, when the ratio of benign and malicious TLS flows is set to 10:1, the detection efficiency of the two-layer framework is 188% faster than that of the single-layer framework, while the average detection accuracy reaches 99.45%.

show abstract

Section: Plos Onementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Two-layer detection framework with a high accuracy and efficiency for a malware family over the TLS protocol

Zheng

Liu

et al. 2020

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…By applying a supervised learning algorithm, the coarse classification model [5][6][7] is used to preprocess network traffic, whereas the fine classification model [13][14][15] is used to identify the type of network traffic accurately. e clustering model [8][9][10][11] based on unsupervised learning algorithms is mainly used to identify unknown network applications and can also be used as a preprocessing method.…”

Section: Related Workmentioning

confidence: 99%

“…However, their impact on the detection effect was not evaluated after carrying out the preprocessing step. Liya [8] used a hierarchical clustering model to preprocess a set of samples. First, they divided the set of samples into multiple clusters, after which they selected several representative samples from each cluster.…”

Section: Unsupervised Learning Modelmentioning

confidence: 99%

“…In recent years, classification and clustering techniques based on machine learning have been widely used in the identification of encrypted communication traffic. Additionally, there have been numerous studies on the application of coarse classification models [5][6][7] and clustering models [8][9][10][11] for preprocessing network traffic. Classification models usually require a large number of labeled samples for training, which results in the improved ability of the classification model to identify the trained samples.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Preprocessing Method for Encrypted Traffic Based on Semisupervised Clustering

Zheng

Liu

Niu

et al. 2020

Security and Communication Networks

View full text Add to dashboard Cite

The explosive growth in network traffic in recent times has resulted in increased processing pressure on network intrusion detection systems. In addition, there is a lack of reliable methods for preprocessing network traffic generated by benign applications that do not steal users’ data from their devices. To alleviate these problems, this study analyzed the differences between benign and malicious traffic produced by benign applications and malware, respectively. To fully express these differences, this study proposed a new set of statistical features for training a clustering model. Furthermore, to mine the communication channels generated by benign applications in batches, a semisupervised clustering method was adopted. Using a small number of labeled samples, our method aggregated historical network traffic into two types of clusters. The cluster that did not contain labeled malicious samples was regarded as a benign traffic cluster. The experimental results were compared using four types of clustering algorithms. The density-based spatial clustering of applications with noise (DBSCAN) clustering algorithm was selected to mine benign communication channels. We also compared our method with two other methods, and the results demonstrated that the benign channels mined through our method were more reliable. Finally, using our method, 1,811 benign transport layer security (TLS) channels were mined from 18,357 TLS communication channels. The number of flows carried by these benign channels comprised 65.37% of the entire network flows, and no malicious flow was included in our results, which proves the effectiveness of our method.

show abstract

Challenge Collapsar (CC) Attack Traffic Detection Based on Packet Field Differentiated Preprocessing and Deep Neural Network

Liu

Zhang

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Hierarchical Clustering Based Network Traffic Data Reduction for Improving Suspicious Flow Detection

Cited by 24 publications

References 18 publications

Two-layer detection framework with a high accuracy and efficiency for a malware family over the TLS protocol

Two-layer detection framework with a high accuracy and efficiency for a malware family over the TLS protocol

Preprocessing Method for Encrypted Traffic Based on Semisupervised Clustering

Challenge Collapsar (CC) Attack Traffic Detection Based on Packet Field Differentiated Preprocessing and Deep Neural Network

Contact Info

Product

Resources

About