Empirical evaluation of hash functions for multipoint measurements

Abstract

A broad spectrum of network measurement applications demand passive multipoint measurements in which data from multiple observation points has to be correlated. Examples are the passive measurement of one-way delay or the observation of the path that a packet takes through a network. Nevertheless, due to high data rates and the need for fine granular measurements, the resource consumption for passive measurements can be immense. Furthermore, the resource consumption depends on the traffic in the network, which usually is highly dynamic. Packet and flow-selection methods provide a solution to reduce and control the resource consumption for passive measurements. In order to apply such techniques to multipoint measurements the selection processes need to be synchronized. Hash-based selection is a deterministic packet selection based on a hash function computed on selected parts of the packet content. This selection decision is consistent throughout the network and enables packet tracing and the measurement of delay between network nodes. Because the selection is based on deterministic function it can introduce bias which leads to wrong estimation of traffic characteristics. In this paper we define a set of quality criteria and select methods to investigate which hash function is most suitable for hash-based packet selection. We analyze 23 non-cryptographic and 2 cryptographic hash functions. Experiments are performed with real traffic traces from different networks. Based on the results we recommend 2 fast hash functions which show low bias and sample a representative subset of the population.

Sampling and Filtering Techniques for IP Packet Selection

“…As shown for instance in [Henk08], the computation times for MD5 and SHA are about 7-10 times higher compared to non-cryptographic functions. The difference increases for small hash input lengths.…”

“…In accordance to considerations in [MoND05] and [Henk08], we define the following desired properties of Hash Functions used for packet selection:…”

“…Comparisons of Hash Functions for packet selection and packet digesting with regard to various criteria can be found in [MoND05] and [Henk08].…”

An Efficient Parallel Top-k Similarity Join for Massive Multidimensional Data Using Spark

“…First, we compute each point's signature S(p). Second, we use BKDRhash [12] function to compute the hash-values of each point's signature S(p) by which all points will be distributed to different buckets. Third, we do the groups of buckets by merging two different buckets to one.…”

“…In this paper, we use the BKDRhash [12] function BH() as the hash method because of its higher speed and less collision. With the hash function, the map(BH) transformation maps the key-value pairs < , S(p i ) > into key-value pairs < (S(p i )), p i >by performing hash calculation over each signature S(p i ) for each point p i .…”