Malicious Traffic Detection with Class Imbalanced Data Based on Coarse-grained Labels

Abstract

In order to resist complex cyber-attacks, a Security Operations Center (SOC) named IHEPSOC has been developed and deployed in the Institute of High Energy Physics (IHEP) of the Chinese Academy of Sciences, which contributed to the reliability and security of the network for IHEP. It has become a major task to integrate state-of-the-art cyber-attack detection methods for IHEPSOC to improve the ability of threat detection. Malicious traffic detection based on machine learning is an emerging security paradigm, which can effectively detect both known and unknown cyberattacks. However, the existing studies usually adopt traditional supervised learning, which often encounter issues when applied to real-world production environment due to its implicit assumptions on the operating dependence. For example, most studies are based on datasets that already have accurate data labels, but labeling these datasets accurately requires significant manual effort. In addition, in the real-world service, the volume of benign traffic data is larger than that of the malicious traffic data, and the imbalance between benign and malicious categories also makes many machine learning detection models difficult to apply to a production environment. Based on these, we propose a detection method for class imbalanced malicious traffic based on coarsegrained data labels, which achieves comparable performance compare to other supervised learning methods. We conducted three experiments, using the Android Malware 2017 dataset, and verified the practicability and effectiveness of the proposed method.

A distributed framework for security operation center in the application of Institute of High Energy Physics

“…These methods apply multiple instance learning to achieve the fine-grained classification of network traffic. The experiment results have demonstrated our detection methods are better than the state-of-the-art detection systems [23,24].…”

“…The AI-based data analysis deploys anomaly detection algorithms and deep neural network to detect attacks. For example, we have developed two weakly-supervised method for malicious network traffic detection [23,24]. These methods apply multiple instance learning to achieve the fine-grained classification of network traffic.…”