Fine-grained tracking of Grid infections

Abstract

Abstract-Previous distributed anomaly detection efforts have operated on summary statistics gathered from each node. This has the advantage that the audit trail is limited in size since event sets can be succinctly represented. While this minimizes the bandwidth consumed and helps scale the detection to a large number of nodes, it limits the infrastructure's ability to identify the source of anomalies. We describe three optimizations that together allow fine-grained tracking of the sources of anomalous activity in a Grid, thereby facilitating precise responses. We demonstrate the scheme's scalability in terms of storage and network bandwidth overhead with an implementation on nodes running BOINC. The results generalize to other types of Grids as well.

Folding and Unfolding Bloom Filters: An Off-Line Planning and On-Line Optimization Problem

“…This can be expressed as follows: given a number of items stored in the Bloom filter, let reduce/increase the size of the Bloom filter by folding/unfolding it so that the false positive rate ρ stays in the following interval [ρ − , ρ + ] and the increase of memory size is kept to a minimum. We acknowledge that halving a Bloom filter was originally suggested in [3] and successfully applied [5] so as to reduce the bandwidth consumption induced by the exchange of Bloom filters in the context of correlated anomaly detection in large-scale grid computing [15]. We herein extend and generalize this approach by introducing the concept of folded and unfolded Bloom filters, which permits highly flexible resizing.…”

“…As pointed out in [3], and promoted and successfully applied in [5], a nice feature of Bloom filters is that they can be halved in size, assuming that the size of the filter is a power of 2. In order to halve a filter, an OR (resp.…”

“…counting Bloom filter) is performed. This approach has been successfully exploited to reduce the bandwidth needed to transmit a Bloom filter [5].…”

Identifying the provenance of correlated anomalies

“…In particular, it can be used as a threat digest [12] with intrusive activity on more than Tc hosts automatically resulting in a digest at all the remaining hosts that allows the same activity to be recognized and flagged before the attack succeeds.…”

“…To support this, the set of all k-tuples observed during this period is logged. We have previously observed that this log grows rapidly [12] but can be effectively represented using a Bloom filter [3]. The i th host Hi will have a Bloom filter Bi to which all k-tuples k1, k2, .…”