Optimal bounds for approximate counting

Nelson, Jelani; Yu, Huacheng

doi:10.48550/arxiv.2010.02116

Cited by 2 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Morris gave a randomized "approximate counter", which allows one to retrieve a constant multiplicative approximation to m with high probability using only O(log log m) bits. The Morris Counter was later analyzed in more detail by Flajolet [21], who showed that O(log log m + log(1/ε) + log(1/δ)) bits of memory are sufficient to return a (1 ± ε) approximation with success probability 1 − δ (this was later improved by Nelson and Yu [139], who showed that O(log log m + log(1/ε) + log log(1/δ)) bits suffice).…”

Section: Entropy Estimationmentioning

confidence: 99%

See 1 more Smart Citation

Binary Hypothesis Testing with Deterministic Finite-Memory Decision Rules

Berg

Ordentlich

Shayevitz

2020

2020 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

The problem of statistical inference in its various forms has been the subject of decades-long extensive research. Most of the effort has been focused on characterizing the behavior as a function of the number of available samples, with far less attention given to the effect of memory limitations on performance. Recently, this latter topic has drawn much interest in the engineering and computer science literature. In this survey paper, we attempt to review the state-of-the-art of statistical inference under memory constraints in several canonical problems, including hypothesis testing, parameter estimation, and distribution property testing/estimation. We discuss the main results in this developing field, and by identifying recurrent themes, we extract some fundamental building blocks for algorithmic construction, as well as useful techniques for lower bound derivations.

show abstract

Section: Entropy Estimationmentioning

confidence: 99%

“…The Morris Counter was later analyzed in more detail by Flajolet [33], who showed that O(log log m+ log(1/ε)+ log(1/δ)) bits of memory are sufficient to return a (1 ± ε) approximation with success probability 1 − δ. A recent result of [34] shows that O(log log m + log(1/ε) + log log(1/δ)) bits suffice for the same task.…”

Section: B Morris Countermentioning

confidence: 99%

Binary Hypothesis Testing with Deterministic Finite-Memory Decision Rules

Berg

Ordentlich

Shayevitz

2020

2020 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

show abstract

“…In a bit more detail, we can use approximate counting to probabilistically estimate the counter up to a small constant factor with probability at least 1 − δ in space O(log(log(n)) + log(log(1/δ)))[43], which explains the discrepancy in dependence on δ in these two equations.…”

mentioning

confidence: 99%

Bounded Memory Active Learning through Enriched Queries

Hopkins¹,

Kane²,

Lovett³

et al. 2021

Preprint

View full text Add to dashboard Cite

The explosive growth of easily-accessible unlabeled data has lead to growing interest in active learning, a paradigm in which data-hungry learning algorithms adaptively select informative examples in order to lower prohibitively expensive labeling costs. Unfortunately, in standard worst-case models of learning, the active setting often provides no improvement over non-adaptive algorithms. To combat this, a series of recent works have considered a model in which the learner may ask enriched queries beyond labels. While such models have seen success in drastically lowering label costs, they tend to come at the expense of requiring large amounts of memory. In this work, we study what families of classifiers can be learned in bounded memory. To this end, we introduce a novel streaming-variant of enriched-query active learning along with a natural combinatorial parameter called lossless sample compression that is sufficient for learning not only with bounded memory, but in a query-optimal and computationally efficient manner as well. Finally, we give three fundamental examples of classifier families with small, easy to compute lossless compression schemes when given access to basic enriched queries: axis-aligned rectangles, decision trees, and halfspaces in two dimensions.

show abstract

Optimal bounds for approximate counting

Cited by 2 publications

References 7 publications

Binary Hypothesis Testing with Deterministic Finite-Memory Decision Rules

Binary Hypothesis Testing with Deterministic Finite-Memory Decision Rules

Bounded Memory Active Learning through Enriched Queries

Contact Info

Product

Resources

About