A formal analysis of conservative update based approximate counting

Einziger, Gil; Friedman, Roy

doi:10.1109/iccnc.2015.7069350

Cited by 14 publications

(21 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Minimal Increment scheme, also known as conservative update, has been proposed as a way to boost the accuracy of Counting Bloom Filters when all the increments are positive [17,26,23]. This technique was successfully used in several fields [46,50,5,32,7,6,41,31,30].…”

Section: Approximate Counting Architecturesmentioning

confidence: 99%

TinyLFU: A Highly Efficient Cache Admission Policy

Einziger

Friedman

2014

2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing

Self Cite

110

View full text Add to dashboard Cite

This paper proposes to use a frequency based cache admission policy in order to boost the effectiveness of caches subject to skewed access distributions. Given a newly accessed item and an eviction candidate from the cache, our scheme decides, based on the recent access history, whether it is worth admitting the new item into the cache at the expense of the eviction candidate.Realizing this concept is enabled through a novel approximate LFU structure called TinyLFU, which maintains an approximate representation of the access frequency of a large sample of recently accessed items. TinyLFU is very compact and light-weight as it builds upon Bloom filter theory.We study the properties of TinyLFU through simulations of both synthetic workloads as well as multiple real traces from several sources. These simulations demonstrate the performance boost obtained by enhancing various replacement policies with the TinyLFU eviction policy. Also, a new combined replacement and eviction policy scheme nicknamed W-TinyLFU is presented. W-TinyLFU is demonstrated to obtain equal or better hit-ratios than other state of the art replacement policies on these traces. It is the only scheme to obtain such good results on all traces.

show abstract

Section: Approximate Counting Architecturesmentioning

confidence: 99%

TinyLFU: A Highly Efficient Cache Admission Policy

Einziger

Friedman

2014

2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing

Self Cite

110

View full text Add to dashboard Cite

show abstract

“…Since an estimate of CUS is always bounded by CMS's estimates from above (and by f x from below), the analysis of CMS holds for CUS as well. We refer the reader to [27] for a refined analysis.…”

Section: Preliminariesmentioning

confidence: 99%

SALSA: Self-Adjusting Lean Streaming Analytics

Basat¹,

Einziger²,

Mitzenmacher³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Counters are the fundamental building block of many data sketching schemes, which hash items to a small number of counters and account for collisions to provide good approximations for frequencies and other measures. Most existing methods rely on fixed-size counters, which may be wasteful in terms of space, as counters must be large enough to eliminate any risk of overflow. Instead, some solutions use small, fixed-size counters that may overflow into secondary structures.This paper takes a different approach. We propose a simple and general method called SALSA for dynamic re-sizing of counters, and show its effectiveness. SALSA starts with small counters, and overflowing counters simply merge with their neighbors. SALSA can thereby allow more counters for a given space, expanding them as necessary to represent large numbers. Our evaluation demonstrates that, at the cost of a small overhead for its merging logic, SALSA significantly improves the accuracy of popular schemes (such as Count-Min Sketch and Count Sketch) over a variety of tasks. Our code is released as open source [1].

show abstract

“…In bioinformatics, we may want to maintain, on the fly, multiplicities of k-mers (words of length k) occurring in a big dataset [20,1,24]. We refer to [10] for more examples of applications.…”

Section: Introductionmentioning

confidence: 99%

“…An attempt to analyse the conservative update algorithm for counting has been made in [10]. Their main idea is a simulation of spectral Bloom filters by a hierarchy of ordinary Bloom filters.…”

Section: Introductionmentioning

confidence: 99%

Analysis of Count-Min sketch under conservative update

Fusy,

Kucherov

2022

Preprint

View full text Add to dashboard Cite

Count-Min sketch is a hash-based data structure to represent a dynamically changing associative array of counters. Here we analyse the counting version of Count-Min under a stronger update rule known as conservative update, assuming the uniform distribution of input keys. We show that the accuracy of conservative update strategy undergoes a phase transition, depending on the number of distinct keys in the input as a fraction of the size of the Count-Min array. We prove that below the threshold, the relative error is asymptotically o(1) (as opposed to the regular Count-Min strategy), whereas above the threshold, the relative error is Θ(1). The threshold corresponds to the peelability threshold of random k-uniform hypergraphs. We demonstrate that even for small number of keys, peelability of the underlying hypergraph is a crucial property to ensure the o(1) error.

show abstract

A formal analysis of conservative update based approximate counting

Cited by 14 publications

References 27 publications

TinyLFU: A Highly Efficient Cache Admission Policy

TinyLFU: A Highly Efficient Cache Admission Policy

SALSA: Self-Adjusting Lean Streaming Analytics

Analysis of Count-Min sketch under conservative update

Contact Info

Product

Resources

About