Higher-Order Asymptotics of Mutual Information for Nonlinear Channels with Non-Gaussian Noise

Prelov, V. V.; Meulen, Edward C. van der

doi:10.1023/b:prit.0000011271.44336.b2

Cited by 3 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Theorem 3. Let X = (X i ) i be any finite set of (dependent) random n-vectors, let X t = (X i,t ) i be defined by (76), and let Z be a white Gaussian random n-vector independent of all other random vectors. If, for any t > 0 and any real-valued coefficients (a i ) i , adding a small perturbation a i Z to the X i,t makes them "more dependent" in the sense that…”

Section: B a Generalized Epi For Dependent Random Vectorsmentioning

confidence: 99%

“…Proof: Let Z be a standard Gaussian random variable independent of X t , and define a = (a i ) i and Y t = X t + √ ε aZ, where aZ = (a i Z) i . The perturbations (76) ensure that the density of X t is smooth, so that the function I(ε) = I{(Y i,t ) i } is differentiable for all ε ≥ 0. Now condition ( 77) is equivalent to the inequality…”

Section: B a Generalized Epi For Dependent Random Vectorsmentioning

confidence: 99%

See 1 more Smart Citation

Information Theoretic Proofs of Entropy Power Inequalities

Rioul

2011

IEEE Trans. Inform. Theory

122

View full text Add to dashboard Cite

While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the EPI hinge on representations of differential entropy using either Fisher information or minimum mean-square error (MMSE), which are derived from de Bruijn's identity. In this paper, we first present an unified view of these proofs, showing that they share two essential ingredients: 1) a data processing argument applied to a covariance-preserving linear transformation; 2) an integration over a path of a continuous Gaussian perturbation. Using these ingredients, we develop a new and brief proof of the EPI through a mutual information inequality, which replaces Stam and Blachman's Fisher information inequality (FII) and an inequality for MMSE by Guo, Shamai and Verd\'u used in earlier proofs. The result has the advantage of being very simple in that it relies only on the basic properties of mutual information. These ideas are then generalized to various extended versions of the EPI: Zamir and Feder's generalized EPI for linear transformations of the random variables, Takano and Johnson's EPI for dependent variables, Liu and Viswanath's covariance-constrained EPI, and Costa's concavity inequality for the entropy power.Comment: submitted for publication in the IEEE Transactions on Information Theory, revised versio

show abstract

Section: B a Generalized Epi For Dependent Random Vectorsmentioning

confidence: 99%

Section: B a Generalized Epi For Dependent Random Vectorsmentioning

confidence: 99%

Information Theoretic Proofs of Entropy Power Inequalities

Rioul

2011

IEEE Trans. Inform. Theory

122

View full text Add to dashboard Cite

show abstract

“…• In [9 [15] for Gaussian noise, [16] for non-Gaussian noise, and [17] for general models under various technical conditions.…”

Section: Remarkmentioning

confidence: 99%

Derivative of Mutual Information at Zero SNR: The Gaussian-Noise Case

Guo

Verdú

2011

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Abstract-Assuming additive Gaussian noise, a general sufficient condition on the input distribution is established to guarantee that the ratio of mutual information to signal-to-noise ratio (SNR) goes to one half nat as SNR vanishes. The result allows SNR-dependent input distribution and side information.Index Terms-Gaussian noise, low-power regime, minimum mean-square error (MMSE), mutual information, signal-to-noise ratio (SNR).

show abstract

“…Η μελέτη της χωρητικότητας μη-γραμμικών καναλιών αποτελεί σημείο έντονου ερευνητικού ενδιαφέροντος εκτός άλλων και στις οπτικές επικοινωνίες [14], όπου η μη-γραμμική φύση της οπτικής ίνας παραμορφώνει τη διάδοση του φωτός. Το πρόβλημα του υπολογισμού της χωρητικότητας μη-γραμμικών καναλιών προσεγγίζεται μέσω της ασυμπτωτικής ανάλυσης υψηλότερων τάξεων της τυχαίας μεταβλητής της αμοιβαίας πληροφορίας [15], [16]. Στη γενική περίπτωση ενός μη-γραμμικού καναλιού, η εύρεση κλειστών τύπων τόσο για τη χωρητικότητα όσο και για εκθέτες τυχαίας κωδικοποίησης παραμένουν ανοιχτά προβλήματα της θεωρίας πληροφορίας.…”

Section: κωδικοποίηση-αποκωδικοποίηση και χωρητικότητα καναλιούunclassified

Θεωρία Πληροφορίας Και Επεξεργασία Σήματος Για (Μη-Γραμμικά) Επικοινωνιακά Κανάλια

Ξενούλης¹

View full text Add to dashboard Cite

Information Theory answers the fundamental problem of communication, that of reproducing at one point either exactly or approximately a message selected at another point. The design and development of synchronous communication systems, like satellite, wireless or optical fibers, rely nowadays in the establishment of performance measure indicators such as the error decoding probability and the channel capacity. Further progress in the field of communications requires among others the development of new, tight performance bounds, that are capable of designating desirable characteristics for coding schemes. The asymptotic behavior of these bounds is not always desirable since they do not reveal practical codes of finite length for simple and more complicated (nonlinear) channel models. Furthemore, the treatment of the inherent nonlinear behaviour of communication systems as well as the calculation of the capacity of the corresponding channels is crucial for enhancing system performance.In this thesis, Gallager's upper bound as well as its variations through the Duman-Salehi bound are improved. The technique that yields this improvement relies on the new inverse exponential sum inequality. The specific criterion is applied to linear codes that are transmitted in discrete, memoryless, linear symmetric channels and upper bounds the word and bit maximum likelihood error decoding probabilities. The analysis designates a new desirable characteristic for linear codes, that is directly connected with the concept of list decoding.The thesis also presents lower bounds on the capacity of nonlinear channels, combining the random coding technique with the theory of martingales. Upper bounds for the maximum likelihood error decoding probability are obtained through the representation of a nonlinear channel with Volterra series. The proposed research follows the main ideas that dominate Shannon's basic work and properly utilizes exponential martingale inequalities in order to bound the probabilities of erroneous decoding regions. The specific analysis is also applied to cases where the noise statistical characteristics (mean value, deviation) remain unknown.A desirable characteristic of the proposed techniques that provide tight bounds for error decoding probability is their ability to designate codes with optimum characteristics and with rates near channel capacity. The present work improves and extends the bound of Shulman-Feder for the family of binary, linear codes that are permutation invariant under list decoding. A new upper bound on list error decoding probability is presented that decreases double exponentially with respect to the code's block length.

show abstract

Higher-Order Asymptotics of Mutual Information for Nonlinear Channels with Non-Gaussian Noise

Cited by 3 publications

References 6 publications

Information Theoretic Proofs of Entropy Power Inequalities

Information Theoretic Proofs of Entropy Power Inequalities

Derivative of Mutual Information at Zero SNR: The Gaussian-Noise Case

Θεωρία Πληροφορίας Και Επεξεργασία Σήματος Για (Μη-Γραμμικά) Επικοινωνιακά Κανάλια

Contact Info

Product

Resources

About