Sparse superposition codes under VAMP decoding with generic rotational invariant coding matrices

Abstract

Sparse superposition codes were originally proposed as a capacity-achieving communication scheme over the gaussian channel, whose coding matrices were made of i.i.d. gaussian entries [1]. We extend this coding scheme to more generic ensembles of rotational invariant coding matrices with arbitrary spectrum, which include the gaussian ensemble as a special case. We further introduce and analyse a decoder based on vector approximate message-passing (VAMP) [2]. Our main findings, based on both a standard replica symmetric potential theory and state evolution analysis, are the superiority of certain structured ensembles of coding matrices (such as partial row-orthogonal) when compared to i.i.d. matrices, as well as a spectrum-independent upper bound on VAMP's threshold. Most importantly, we derive a simple "spectral criterion" for the scheme to be at the same time capacity-achieving while having the best possible algorithmic threshold, in the "large section size" asymptotic limit. Our results therefore provide practical design principles for the coding matrices in this promising communication scheme.

The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?

“…Then, by using (54) with i = t + 1 and Proposition 2, we obtain that ( 62) and ( 63) hold, thus concluding the inductive proof. The result we have just proved by induction, combined with (47), gives that (45) By combining ( 69) and ( 71), we obtain that the desired result (19) holds, which concludes the proof. By additivity of the R-transform for (asymptotically) free random matrices [70], denoting R t (x) the R-transform of Z t , we obtain…”

“…The case in which Z is actually Gaussian has been thoroughly studied [53,28,55,54,12,61]. Beyond Gaussianity, a rapidly growing literature is focusing on rotationally invariant models assuming perfect knowledge of the statistics of the structured matrix appearing in the problem (such as noise in inference, a sensing, data, or coding matrix in regression tasks, weight matrices in neural networks, or a matrix of interactions in spin glass models) [67,23,37,38,42,65,56,57,74,77,78,16,33,66,71,47]. However, despite this impressive progress when the noise statistics is known, low-rank estimation in a mismatched setting with partial to no knowledge of the statistics of the rotationally invariant noise matrix remains poorly understood.…”

Exponentially Fast Concentration of Vector Approximate Message Passing to its State Evolution

“…Gaussian measurement matrices, have found many applications in the literature to date. These algorithms play an important role in high-dimensional statistics [10,12,41], wireless communications [7,22], and many other applications [1,30,31,32,42,44]. In particular, the finite sample analysis for VAMP and GAMP presented here can be used to find the error rates for the capacityachieving sparse regression coding schemes introduced in [7,22], which use VAMP and GAMP as decoders.…”

“…In particular, there has been no concentration results developed for generalized versions of AMP, which can handle models such as (1.1), or for AMP algorithms with measurement matrices that are not i.i.d. Gaussian, despite such algorithms playing an important role in high-dimensional statistics [10,12,41], wireless communications [7,22], and many other applications [1,30,31,32,42,44].…”

“…We emphasize that while refining earlier asymptotic results to provide non-asymptotic guarantees is useful to clarify the effect of the iteration number on the problem dimension and the sped of convergence to the state evolution predictions, it also has other application-specific implications; e.g. the finite sample analyses of [38] were used to establish the error exponent for sparse regression codes with AMP decoding for the additive white Gaussian noise channel [39] and the results in this work will be similarly useful in establishing error exponent for sparse regression codes with GAMP or VAMP decoding for more general channel models considered in [7,22].…”