FARE: Enabling Fine-grained Attack Categorization under Low-quality Labeled Data

Abstract

Supervised machine learning classifiers have been widely used for attack detection, but their training requires abundant high-quality labels. Unfortunately, high-quality labels are difficult to obtain in practice due to the high cost of data labeling and the constant evolution of attackers. Without such labels, it is challenging to train and deploy targeted countermeasures. In this paper, we propose FARE, a clustering method to enable fine-grained attack categorization under low-quality labels. We focus on two common issues in data labels: 1) missing labels for certain attack classes or families; and 2) only having coarsegrained labels available for different attack types. The core idea of FARE is to take full advantage of the limited labels while using the underlying data distribution to consolidate the lowquality labels. We design an ensemble model to fuse the results of multiple unsupervised learning algorithms with the given labels to mitigate the negative impact of missing classes and coarsegrained labels. We then train an input transformation network to map the input data into a low-dimensional latent space for fine-grained clustering. Using two security datasets (Android malware and network intrusion traces), we show that FARE significantly outperforms the state-of-the-art (semi-)supervised learning methods in clustering quality/correctness. Further, we perform an initial deployment of FARE by working with a large e-commerce service to detect fraudulent accounts. With realworld A/B tests and manual investigation, we demonstrate the effectiveness of FARE to catch previously-unseen frauds. Proposed Solution. In this paper, we aim to enable finegrained attack categorization using low-quality labels. The goal is to discover the clustering structures in the data to assist human analysts to derive high-quality labels. We propose FARE, a semi-supervised method to address the issues of both missing classes and coarse-grained labels in poorlylabeled datasets. At the high-level, FARE's input is a dataset where only a small portion of the data is labeled, and the labels are of a low-quality. After running FARE, it outputs the clustering assignment for all the data samples. The data samples are expected to be either correctly clustered under the known labels or form new groups to represent the new labels. By correctly recovering the clustering structures in the input dataset, FARE provides the much-needed support for human analysts to generate high-quality labels. The core idea of FARE is to take full advantage of the limited labels while using the underlying data distribution to

Semi-WTC: A Practical Semi-supervised Framework for Attack Categorization through Weight-Task Consistency

“…In our experiments, FARE [10], ACID [11], and traditional machine learning algorithms are chosen as baseline methods for comparison. FARE and ACID are proposed recently in 2020 and 2021 with high citations.…”

“…Machine learning-based attack detection can be divided into coarse-grained attack detection [6] [7] and fine-grained attack detection [10] [11]. DiFF-RF [6] introduced an ensemble approach composed of random partitioning binary trees 1. https://github.com/HUANGLIZI/WTC.…”

“…to detect anomalies, while Whisper [7] introduced a realtime machine learning-based malicious traffic detection system thorough utilizing sequential information represented by the frequency domain features to achieve bounded information loss, as well as ensuring high detection accuracy. In the field of cyber security, FARE [10] utilizes eans clustering as an unsupervised classification algorithm supplemented with partially labeled data for semi-supervised learning. Recently, Diallo [11] proposed an intrusion detection model based on adaptive clustering (ACID), which uses lowdimensional embeddings learned by efficient convolutional neural networks to solve the problem that traditional classification models are sensitive to feature changes and can effectively separate samples from different classes.…”

“…Nowadays, machine learning-based network intrusion detection algorithms [4] [5] that can detect unseen attacks are flourishing. Existing methods can be divided into two categories: coarse-grained attack detection [6] [7] [8] [9] and fine-grained attack detection [10] [11]. Coarse-grained methods only distinguish between attack and normal, which are not well suited for detailed analysis of attacks.…”

DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications

“…Improving DL-based security systems. Recently, several works have been proposed for improving the practicality of (unsupervised) DL-based security systems, such as improving the robustness of DLbased systems [17,21], performing lifelong learning [11], detecting concept drift [58], learning from low-quality labeled data [30]. They are orthogonal to our work and could be adopted together for developing more practical security systems.…”