Binomial and Poisson distributions as maximum entropy distributions

Harremoës, Peter

doi:10.1109/18.930936

Cited by 115 publications

(92 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yasaee for their helpful comments. We are specifically indebted to the two anonymous reviewers and the associate editor for their helpful comments; in fact, Dr. Gerhard Kramer has brought to our attentions the similarity of our upper bound for the noiseless case with that of T-codes developed by Chang and Weldon [24], and the proof that our conjectured upper bound is a true upper bound for the noiseless case (Section 4) based on the works of Shepp and Olgin [27] and the two corresponding papers [28]- [29].…”

Section: Acknowledgementmentioning

confidence: 70%

Bounds on the Sum Capacity of Synchronous Binary CDMA Channels

Alishahi

Marvasti

Aref

et al. 2009

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

In this paper, we obtain a family of lower bounds for the sum capacity of Code Division Multiple Access (CDMA) channels assuming binary inputs and binary signature codes in the presence of additive noise with an arbitrary distribution. The envelope of this family gives a relatively tight lower bound in terms of the number of users, spreading gain and the noise distribution. The derivation methods for the noiseless and the noisy channels are different but when the noise variance goes to zero, the noisy channel bound approaches the noiseless case. The behavior of the lower bound shows that for small noise power, the number of users can be much more than the spreading gain without any significant loss of information (overloaded CDMA). A conjectured upper bound is also derived under the usual assumption that the users send out equally likely binary bits in the presence of additive noise with an arbitrary distribution. As the noise level increases, and/or, the ratio of the number of users and the spreading gain increases, the conjectured upper bound approaches the lower bound. We have also derived asymptotic limits of our bounds that can be compared to a formula that Tanaka obtained using techniques from statistical physics; his bound is close to that of our conjectured upper bound for large scale systems.

show abstract

Section: Acknowledgementmentioning

confidence: 70%

Bounds on the Sum Capacity of Synchronous Binary CDMA Channels

Alishahi

Marvasti

Aref

et al. 2009

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…Put f (k) = log (Pr (P o (λ in ) + Y = k)) . All Bernoulli sums are log-concave [12] and therefore the same property holds for generalized Bernoulli sums. According to [14,Theorem 6.1] it is sufficient to prove that E (f (X)) is minimal for the Poisson distribution, but this follows from Lemma 3.…”

Section: Proof Rewrite the Mutual Information Asmentioning

confidence: 82%

A Nash Equilibrium related to the Poisson Channel

Harremoës¹,

Vignat²

2003

Communications in Information and Systems

View full text Add to dashboard Cite

show abstract

“…, n}. Under the constraint that n i=1 E[X i ] ≤ λ, it follows from the maximal entropy result in [11], [14] and [25] that the entropy of Y is maximized when the n independent inputs are i.i.d. with mean p = λ n , and consequently the channel output Y is Binomially distributed with Y ∼ Binom n, λ n .…”

Section: Ieee International Symposium On Information Theorymentioning

confidence: 99%

Entropy bounds for discrete random variables via coupling

Sason

2013

2013 IEEE International Symposium on Information Theory

View full text Add to dashboard Cite

This work provides new bounds on the difference between the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal couplings, and the bounds apply to discrete random variables which are defined over finite or countably infinite alphabets. Loosened versions of these bounds are demonstrated to reproduce some previously reported results. The use of the new entropy bounds is exemplified for the Poisson approximation, where bounds on the local and total variation distances follow from Stein's method. The full paper version for this work is available at http://arxiv.org/abs/1209.5259.

show abstract

Binomial and Poisson distributions as maximum entropy distributions

Abstract: The binomial and the Poisson distributions are shown to be maximum entropy distributions of suitably defined sets. Poisson's law is considered as a case of entropy maximization, and also convergence in information divergence is established.

Cited by 115 publications

References 13 publications

Bounds on the Sum Capacity of Synchronous Binary CDMA Channels

Bounds on the Sum Capacity of Synchronous Binary CDMA Channels

A Nash Equilibrium related to the Poisson Channel

Entropy bounds for discrete random variables via coupling

Contact Info

Product

Resources

About