Casting out Demons: Sanitizing Training Data for Anomaly Sensors

Abstract

The efficacy of Anomaly Detection (AD) sensors depends heavily on the quality of the data used to train them. Artificial or contrived training data may not provide a realistic view of the deployment environment. Most realistic data sets are dirty; that is, they contain a number of attacks or anomalous events. The size of these high-quality training data sets makes manual removal or labeling of attack data infeasible. As a result, sensors trained on this data can miss attacks and their variations. We propose extending the training phase of AD sensors (in a manner agnostic to the underlying AD algorithm) to include a sanitization phase. This phase generates multiple models conditioned on small slices of the training data. We use these "micromodels" to produce provisional labels for each training input, and we combine the micro-models in a voting scheme to determine which parts of the training data may represent attacks. Our results suggest that this phase automatically and significantly improves the quality of unlabeled training data by making it as "attack-free" and "regular" as possible in the absence of absolute ground truth. We also show how a collaborative approach that combines models from different networks or domains can further refine the sanitization process to thwart targeted training or mimicry attacks against a single site.

Cross-Domain Collaborative Anomaly Detection: So Far Yet So Close

“…Averaged over eight weeks both sites keep over 40% of bits in common while in the three week run this is closer to 50%. This reinforces existing work [5] showing that traffic patterns do evolve over time indicating that updating normal models periodically should increase effectiveness. With our three week data set, we also have an additional web server from one administrative domain.…”

“…Alexsander Lazarevic et al compares several AD systems in Network Intrusion Detection [12]. For our analysis, we use the STAND [5] method and Anagram [30] CAD sensor as our base CAD system. The STAND process shows improved results for CAD sensors by introducing a sanitization phase to scrub training data.…”

“…In our experiments, we leverage the STAND [5] optimizations of the Anagram [30] CAD sensor although any CAD sensor with a high detection rate could be used with our approach. However, we apply the CAD sensors on normalized input instead of full packet content as they originally operated on in order to obtain more accurate results.…”

“…This provides space efficiency and incredible speed suitable for high speed networks since adding an element or checking if one is already present are constant time operations. Each normalized content is spilt into 5-gram sections as in [5] using a sliding window of five characters. See Figure 3(a) for an example.…”

Pattern Recognition Systems under Attack

“…To significantly compromise the training phase of a learning algorithm, an attack has to be exhibit some characteristics that are different from those shown by the rest of the training data, otherwise it would have no impact at all. Therefore, most of the training attacks can be regarded as outliers, and countered either by data sanitization (i.e., outlier detection) [28] or by exploiting robust statistics [40,53] to mitigate the outliers' impact on learning (e.g., robust principal component analysis [66,29]). Notably, in [27] the robustness of SVMs to training data contamination has been formally analyzed under the framework of Robust Statistics [40,53], highlighting that bounded kernels and bounded loss functions may significantly limit the outliers' impact on classifier training.…”