Empirical Evaluation of Hash Functions for PacketID Generation in Sampled Multipoint Measurements

Henke, Christian; Schmoll, Carsten; Zseby, Tanja

doi:10.1007/978-3-642-00975-4_20

Cited by 15 publications

(10 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that this comparison strictly depends on the specific filter size employed; large filters would yield further performance reduction for the windowbased construction, whose performance linearly depends on plete telephony traffic in most operator network. This is a very promising result, as our SW implementation is still far from being optimized and, particularly, it is still based on a slow cryptographic hash (SHA-1, which performs at least one to two orders of magnitude less than state of the art non cryptographic hashes [14]). Figures 1 and 2 show how close the stream operation of VoIPSTREAM succeeds in approaching the VoIP SEAL's classification performance.…”

Section: Experimental Evaluationmentioning

confidence: 98%

On-demand time-decaying bloom filters for telemarketer detection

Bianchi

d’Heureuse²,

Niccolini³

2011

SIGCOMM Comput. Commun. Rev.

View full text Add to dashboard Cite

Several traffic monitoring applications may benefit from the availability of efficient mechanisms for approximately tracking smoothed time averages rather than raw counts. This paper provides two contributions in this direction. First, our analysis of Time-decaying Bloom filters, formerly proposed data structures devised to perform approximate Exponentially Weighted Moving Averages on streaming data, reveals two major shortcomings: biased estimation when measurements are read in arbitrary time instants, and slow operation resulting from the need to periodically update all the filter's counters at once. We thus propose a new construction, called On-demand Time-decaying Bloom filter, which relies on a continuous-time operation to overcome the accuracy/performance limitations of the original window-based approach. Second, we show how this new technique can be exploited in the design of high performance stream-based monitoring applications, by developing VoIPSTREAM, a proof-of-concept real-time analysis version of a formerly proposed system for telemarketing call detection. Our validation results, carried out over real telephony data, show how VoIPSTREAM closely mimics the feature extraction process and traffic analysis techniques implemented in the offline system, at a significantly higher processing speed, and without requiring any storage of per-user call detail records.

show abstract

Section: Experimental Evaluationmentioning

confidence: 98%

On-demand time-decaying bloom filters for telemarketer detection

Bianchi

d’Heureuse²,

Niccolini³

2011

SIGCOMM Comput. Commun. Rev.

View full text Add to dashboard Cite

show abstract

“…Henke et al [16] have extensively studied the collision performance of a set of hash functions and found that the Bob hash function overall provides the best performance. In this paper, we choose the Bob hash function to create the identifier of each packet and to compute the bucket indexes.…”

Section: A Architecturementioning

confidence: 99%

Every Timestamp Counts: Accurate Tracking of Network Latencies Using Reconcilable Difference Aggregator

Barlet‐Ros

2018

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

User-facing services deployed in data centers must respond quickly to user actions. The measurement of network latencies is of paramount importance. Recently, a new family of compact data structures has been proposed to estimate one-way latencies. In order to achieve scalability, these new methods rely on timestamp aggregation. Unfortunately, this approach suffers from serious accuracy problems in the presence of packet loss and reordering, given that a single lost or out-of-order packet may invalidate a huge number of aggregated samples.In this paper, we unify the problem to detect lost and reordered packets within the set reconciliation framework. Although the set reconciliation approach and the data structures for aggregating packet timestamps are previously known, the combination of these two principles is novel. We present a space-efficient synopsis called Reconcilable Difference Aggregator (RDA). RDA maximizes the percentage of useful packets for latency measurement by mapping packets to multiple banks and repairing aggregated samples that have been damaged by lost and reordered packets. RDA simultaneously obtains the average and the standard deviation of the latency. We provide a formal guarantee of the performance and derive optimized parameters. We further design and implement a user-space passive latency measurement system that addresses practical issues of integrating RDA into the network stack. Our extensive evaluation shows that compared to existing methods, our approach improves the relative error of the average latency estimation in 10-15 orders of magnitude, and the relative error of the standard deviation in 0.5-6 orders of magnitude.

show abstract

“…Hash functions in these applications are designed to be as uniform as possible to avoid collisions. Numerous empirical studies establish that many commonly applied hash functions (e.g., Linear, SDBM, MD5, SHA) have negligible differences with a true uniform random variable [25,32]. Cryptographic hashes work particularly well and are supported by most commercial and open source systems, for example MySQL provides MD5 and SHA1.…”

Section: Hashing Operatormentioning

confidence: 99%

Stale view cleaning

et al. 2015

View full text Add to dashboard Cite

Materialized views (MVs), stored pre-computed results, are widely used to facilitate fast queries on large datasets. When new records arrive at a high rate, it is infeasible to continuously update (maintain) MVs and a common solution is to defer maintenance by batching updates together. Between batches the MVs become increasingly stale with incorrect, missing, and superfluous rows leading to increasingly inaccurate query results. We propose Stale View Cleaning (SVC) which addresses this problem from a data cleaning perspective. In SVC, we efficiently clean a sample of rows from a stale MV, and use the clean sample to estimate aggregate query results. While approximate, the estimated query results reflect the most recent data. As sampling can be sensitive to long-tailed distributions, we further explore an outlier indexing technique to give increased accuracy when the data distributions are skewed. SVC complements existing deferred maintenance approaches by giving accurate and bounded query answers between maintenance. We evaluate our method on a generated dataset from the TPC-D benchmark and a real video distribution application. Experiments confirm our theoretical results: (1) cleaning an MV sample is more efficient than full view maintenance, (2) the estimated results are more accurate than using the stale MV, and (3) SVC is applicable for a wide variety of MVs.

show abstract

Empirical Evaluation of Hash Functions for PacketID Generation in Sampled Multipoint Measurements

Cited by 15 publications

References 5 publications

On-demand time-decaying bloom filters for telemarketer detection

On-demand time-decaying bloom filters for telemarketer detection

Every Timestamp Counts: Accurate Tracking of Network Latencies Using Reconcilable Difference Aggregator

Stale view cleaning

Contact Info

Product

Resources

About