A Simple Proof for the Optimality of Randomized Posterior Matching

Shayevitz, Ofer; Feder, Meir

doi:10.1109/tit.2016.2549043

Cited by 17 publications

(12 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Convex functionals on the probability simplex such as entropy, KL divergence, etc. have been shown to be useful for analyzing adaptive systems [2], [29], [30], [31] with the Bayes' rule dynamics on the posterior distribution. Particularly, the functional average log-likelihood [30] has shown its usefulness in analyzing the behaviour of the posterior in feedback coding systems [28], dynamic spectrum sensing [10], hypothesis testing [32], active learning [32], etc.…”

Section: Appendix B Average Log-likelihood and The Extrinsic Jensen-smentioning

confidence: 99%

Active Learning and CSI Acquisition for mmWave Initial Alignment

Chiu

Ronquillo

Javidi

2019

IEEE J. Select. Areas Commun.

106

133

View full text Add to dashboard Cite

Millimeter wave (mmWave) communication with large antenna arrays is a promising technique to enable extremely high data rates due to large available bandwidth in mmWave frequency bands. In addition, given the knowledge of an optimal directional beamforming vector, large antenna arrays have been shown to overcome both the severe signal attenuation in mmWave as well as the interference problem. However, fundamental limits on achievable learning rate of an optimal beamforming vector remain.This paper considers the problem of adaptive and sequential optimization of the beamforming vectors during the initial access phase of communication. With a single-path channel model, the problem is reduced to actively learning the Angle-of-Arrival (AoA) of the signal sent from the user to the Base Station (BS). Drawing on the recent results in the design of a hierarchical beamforming codebook [1], sequential measurement dependent noisy search strategies [2], and active learning from an imperfect labeler [3], an adaptive and sequential alignment algorithm is proposed.For any given resolution and error probability of the estimated AoA, an upper bound on the expected search time of the proposed algorithm is derived via Extrinsic Jensen-Shannon Divergence. The upper bound demonstrates that the search time of the proposed algorithm asymptotically matches the performance of the noiseless bisection search up to a constant factor, in effect, characterizing the AoA acquisition rate. Furthermore, the upper bound shows that the acquired AoA error probability decays exponentially fast with the search time with an exponent that is a decreasing function of the acquisition rate.Numerically, the proposed algorithm is compared with prior work where a significant improvement of the system communication rate is observed. Most notably, in the relevant regime of low (−10dB to +5dB) raw SNR, this establishes the first practically viable solution for initial access and, hence, the first demonstration of stand-alone mmWave communication.

show abstract

Section: Appendix B Average Log-likelihood and The Extrinsic Jensen-smentioning

confidence: 99%

Active Learning and CSI Acquisition for mmWave Initial Alignment

Chiu

Ronquillo

Javidi

2019

IEEE J. Select. Areas Commun.

106

133

View full text Add to dashboard Cite

show abstract

“…Li and El Gamal [25] considered a non-sequential, fixed-rate, fixed-block-length feedback coding scheme for discrete memoryless channels (DMCs) when W = (0, 1) and introduced a random dither known to the encoder and decoder to provide a simple proof of achieving capacity. Also, Shayevitz and Feder [26] examined a randomized variant of the original PM scheme with a random dither and were able to provide a much simpler proof of optimality over general memoryless channels when W ∈ (0, 1). That is, as compared to the proofs above which are restricted to non-sequential variants with a fixed number of messages and only apply to DMCs, Shayevitz and Feder [26] used a random dither to provide a proof of optimality of the sequential, horizon-free, randomized posterior matching scheme over general memoryless channels.…”

Section: A Previous and Related Workmentioning

confidence: 99%

“…Also, Shayevitz and Feder [26] examined a randomized variant of the original PM scheme with a random dither and were able to provide a much simpler proof of optimality over general memoryless channels when W ∈ (0, 1). That is, as compared to the proofs above which are restricted to non-sequential variants with a fixed number of messages and only apply to DMCs, Shayevitz and Feder [26] used a random dither to provide a proof of optimality of the sequential, horizon-free, randomized posterior matching scheme over general memoryless channels. These settings where the encoder and decoder share a common source of randomness by way of a dither, however, may be undesirable in some situations (e.g.…”

Section: A Previous and Related Workmentioning

confidence: 99%

Construction and Analysis of Posterior Matching in Arbitrary Dimensions via Optimal Transport

Mesa,

Ma,

Gorantla

et al. 2019

Preprint

View full text Add to dashboard Cite

The posterior matching scheme, for feedback encoding of a message point lying on the unit interval over memoryless channels, maximizes mutual information for an arbitrary number of channel uses. However, it in general does not always achieve any positive rate; so far, elaborate analyses have been required to show that it achieves any positive rate below capacity. More recent efforts have introduced a random "dither" shared by the encoder and decoder to the problem formulation, to simplify analyses and guarantee that the randomized scheme achieves any rate below capacity. Motivated by applications (e.g. human-computer interfaces) where (a) common randomness shared by the encoder and decoder may not be feasible and (b) the message point lies in a higher dimensional space, we focus here on the original formulation without common randomness, and use optimal transport theory to generalize the scheme for a message point in a higher dimensional space. By defining a stricter, almost sure, notion of message decoding, we use classical probabilistic techniques (e.g. change of measure and martingale convergence) to establish succinct necessary and sufficient conditions on when the message point can be recovered from infinite observations: Birkhoff ergodicity of a random process sequentially generated by the encoder. We also show a surprising "all or nothing" result: the same ergodicity condition is necessary and sufficient to achieve any rate below capacity. We provide applications of this message point framework in human-computer interfaces and multi-antenna communications.

show abstract

“…This upper bound is trivially an upper bound also for the feedback model, since non causal knowledge might increase the capacity only. Then, we construct a simple coding scheme for the feedback setting, inspired by the posterior matching principle [15]- [18]. The coding scheme enables both the encoder and the decoder to systematically reduce the size of the set of possible messages to a single message, which is then declared by the decoder as the correct message.…”

Section: Introductionmentioning

confidence: 99%

Feedback capacity and coding for the (0, k)-RLL input-constrained BEC

Peled

Sabag

Permuter

2017

2017 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

The input-constrained binary erasure channel (BEC) with strictly causal feedback is studied. The channel input sequence must satisfy the (0, k)-runlength limited (RLL) constraint, i.e., no more than k consecutive '0's are allowed. The feedback capacity of this channel is derived for all k ≥ 1, and is given bywhere ε is the erasure probability, ε = 1 − ε and H 2 (·) is the binary entropy function. The maximization is only over δ k−1 , while the parameters δ i for i ≤ k − 2 are straightforward functions of δ k−1 . The lower bound is obtained by constructing a simple coding for all k ≥ 1. It is shown that the feedback capacity can be achieved using zero-error, variable length coding. For the converse, an upper bound on the non-causal setting, where the erasure is available to the encoder just prior to the transmission, is derived. This upper bound coincides with the lower bound and concludes the search for both the feedback capacity and the non-causal capacity. As a result, non-causal knowledge of the erasures at the encoder does not increase the feedback capacity for the (0, k)-RLL input-constrained BEC. This property does not hold in general: the (2, ∞)-RLL input-constrained BEC, where every '1' is followed by at least two '0's, is used to show that the feedback capacity can be strictly greater than the non-causal capacity.

show abstract

A Simple Proof for the Optimality of Randomized Posterior Matching

Cited by 17 publications

References 13 publications

Active Learning and CSI Acquisition for mmWave Initial Alignment

Active Learning and CSI Acquisition for mmWave Initial Alignment

Construction and Analysis of Posterior Matching in Arbitrary Dimensions via Optimal Transport

Feedback capacity and coding for the (0, k)-RLL input-constrained BEC

Contact Info

Product

Resources

About