Efficient compression of monotone and m-modal distributions

Acharya, Jayadev; Jafarpour, Ashkan; Orlitsky, Alon; Suresh, Ananda Theertha

doi:10.1109/isit.2014.6875157

Cited by 5 publications

(4 citation statements)

References 26 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [Bir87], Birgé showed that any monotone distribution p over [n] can be obliviously decomposed into O(log(n)/ε) intervals, such that the flattening p (recall Definition 6) of p over these intervals is ε-close to p in total variation distance. [AJOS14] extend this result, giving a bound between the χ 2 -distance of p and p. We strengthen these results by extending them to monotone distributions over [n] d . In particular, we partition the domain [n] d of p into O((d log(n)/ε 2 ) d ) rectangles, and compare it with p, the flattening over these rectangles.…”

Section: Testing Monotonicitysupporting

confidence: 64%

“…Birgé [Bir87] showed that any monotone distribution is estimated to a total variation ε with a O(log(n)/ε)-piecewise constant distribution. Moreover, the intervals over which the output is constant is independent of the distribution p. This result, was strengthened to the Kullback-Leibler divergence by [AJOS14] to study the compression of monotone distributions. They upper bound the KL divergence by χ 2 distance and then bound the χ 2 distance.…”

Section: C1 a Structural Results For Monotone Distributions On The Hy...mentioning

confidence: 90%

See 1 more Smart Citation

Optimal Testing for Properties of Distributions

Acharya,

Daskalakis,

Kamath

2015

Preprint

Self Cite

View full text Add to dashboard Cite

Given samples from an unknown distribution p, is it possible to distinguish whether p belongs to some class of distributions C versus p being far from every distribution in C? This fundamental question has received tremendous attention in statistics, focusing primarily on asymptotic analysis, and more recently in information theory and theoretical computer science, where the emphasis has been on small sample size and computational complexity. Nevertheless, even for basic properties of distributions such as monotonicity, log-concavity, unimodality, independence, and monotone-hazard rate, the optimal sample complexity is unknown.We provide a general approach via which we obtain sample-optimal and computationally efficient testers for all these distribution families. At the core of our approach is an algorithm which solves the following problem: Given samples from an unknown distribution p, and a known distribution q, are p and q close in χ 2 -distance, or far in total variation distance? With this tool in place, we develop a general testing framework which leads to the following results: * Supported by a grant from MITEI-Shell program.

show abstract

Section: Testing Monotonicitysupporting

confidence: 64%

Section: C1 a Structural Results For Monotone Distributions On The Hy...mentioning

confidence: 90%

Optimal Testing for Properties of Distributions

Acharya,

Daskalakis,

Kamath

2015

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The second approach restricted the class of distributions compressed. A series of works studied class of monotone distributions Shamir [2013], Acharya et al [2014a]. Recently, Acharya et al [2014a] showed that the class M k of monotone distributions over {1, .…”

Section: Previous Resultsmentioning

confidence: 99%

Universal compression of power-law distributions

Falahatgar

Jafarpour

Orlitsky

et al. 2015

2015 IEEE International Symposium on Information Theory (ISIT)

Self Cite

View full text Add to dashboard Cite

English words and the outputs of many other natural processes are well-known to follow a Zipf distribution. Yet this thoroughly-established property has never been shown to help compress or predict these important processes. We show that the expected redundancy of Zipf distributions of order α > 1 is roughly the 1/α power of the expected redundancy of unrestricted distributions. Hence for these orders, Zipf distributions can be better compressed and predicted than was previously known. Unlike the expected case, we show that worstcase redundancy is roughly the same for Zipf and for unrestricted distributions. Hence Zipf distributions have significantly different worst-case and expected redundancies, making them the first natural distribution class shown to have such a difference.

show abstract

“…[19] studied the class M n k of length−n sequences from monotone distributions over [k], and tightly characterized the redundancy for any k = O(n). Recently, [3] studied M n k for much larger range of k and in particular showed that the class is universally compressible for all k = exp(o(n log n)) and is not universally compressible for any k = exp(Ω(n)).…”

Section: Introductionmentioning

confidence: 99%

Universal Compression of Envelope Classes: Tight Characterization via Poisson Sampling

Acharya,

Jafarpour,

Orlitsky

et al. 2014

Preprint

Self Cite

View full text Add to dashboard Cite

The Poisson-sampling technique eliminates dependencies among symbol appearances in a random sequence. It has been used to simplify the analysis and strengthen the performance guarantees of randomized algorithms. Applying this method to universal compression, we relate the redundancies of fixed-length and Poisson-sampled sequences, use the relation to derive a simple single-letter formula that approximates the redundancy of any envelope class to within an additive logarithmic term. As a first application, we consider i.i.d. distributions over a small alphabet as a step-envelope class, and provide a short proof that determines the redundancy of discrete distributions over a small alphabet up to the first order terms. We then show the strength of our method by applying the formula to tighten the existing bounds on the redundancy of exponential and power-law classes, in particular answering a question posed by Boucheron, Garivier and Gassiat [6].

show abstract

Efficient compression of monotone and m-modal distributions

Abstract: We consider universal compression of n samples drawn independently according to a monotone or m-modal distribution over k elements. We show that for all these distributions, the per-sample redundancy diminishes to 0 if k = exp(o(n/ log n)) and is at least a constant if k = exp(Ω(n)).

Cited by 5 publications

References 26 publications

Optimal Testing for Properties of Distributions

Optimal Testing for Properties of Distributions

Universal compression of power-law distributions

Universal Compression of Envelope Classes: Tight Characterization via Poisson Sampling

Contact Info

Product

Resources

About