Sufficient Statistic Memory AMP

Liu, Lei; Huang, Shunqi; Kurkoski, Brian M.

doi:10.48550/arxiv.2112.15327

Cited by 3 publications

(5 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is proved that the SE of OAMP is convergent [41], [42]. Let (ρ * , v * ) with v * = φS SE (ρ * ) be the unique fixed point.…”

Section: Provides a Graphical Illustration Of (11)mentioning

confidence: 99%

Capacity Optimality of OAMP in Coded Large Unitarily Invariant Systems

Liu¹,

Liang²,

Li³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper investigates a large unitarily invariant system (LUIS) involving a unitarily invariant sensing matrix, an arbitrary fixed signal distribution, and forward error control (FEC) coding. Several area properties are established based on the state evolution of orthogonal approximate message passing (OAMP) in an un-coded LUIS. Under the assumptions that the state evolution for joint OAMP and FEC decoding is correct and the replica method is reliable, we analyze the achievable rate of OAMP. We prove that OAMP reaches the constrained capacity predicted by the replica method of the LUIS with an arbitrary signal distribution based on matched FEC coding. Meanwhile, we elaborate a constrained capacity-achieving coding principle for LUIS, based on which irregular low-density paritycheck (LDPC) codes are optimized for binary signaling in the simulation results. We show that OAMP with the optimized codes has significant performance improvement over the un-optimized ones and the well-known Turbo linear MMSE algorithm. For quadrature phase-shift keying (QPSK) modulation, constrained capacity-approaching bit error rate (BER) performances are observed under various channel conditions.

show abstract

“…It is proved that the SE of OAMP is convergent [41], [42]. Let (ρ * , v * ) with v * = φS SE (ρ * ) be the unique fixed point.…”

Section: Provides a Graphical Illustration Of (11)mentioning

confidence: 99%

Capacity Optimality of OAMP in Coded Large Unitarily Invariant Systems

Liu¹,

Liang²,

Li³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Assumption 1: The entries of x are IID with zero mean. The variance of x is normalized, i.e., 1 N E{ x 2 } a.s.…”

Section: A Problem Formulationmentioning

confidence: 99%

“…We assume that M,N → ∞ with a fixed δ = M/N , and A is known and is right-unitarily-invariant, i.e., U Σ and V are independent, and V is Haar distributed (or equivalently, isotropically random). Furthermore, the empirical eigenvalue distribution of AA H converges almost surely to a compactly supported deterministic distribution with unit first moment in the large system limit, i.e., 1 N tr{AA H } a.s.…”

Section: A Problem Formulationmentioning

confidence: 99%

“…In [1], using the sufficient statistic technique, we reveal two interesting properties of OAMP/VAMP: 1) the covariance matrices are L-banded and are convergent, and 2) damping and memory are useless. In [1], we also prove that SS-MAMP has a faster convergence speed than the MAMP in [22], [23].…”

Section: A Definition and Properties Of Ss-mamp Definition 4 (Suffici...mentioning

confidence: 99%

See 1 more Smart Citation

Sufficient Statistic Memory Approximate Message Passing

Liu¹,

Huang²,

Kurkoski³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Approximate message passing (AMP) type algorithms have been widely used in the signal reconstruction of certain large random linear systems. A key feature of the AMPtype algorithms is that their dynamics can be correctly described by state evolution. However, state evolution does not necessarily guarantee the convergence of iterative algorithms. To solve the convergence problem of AMP-type algorithms in principle, this paper proposes a memory AMP (MAMP) under a sufficient statistic condition, named sufficient statistic MAMP (SS-MAMP). We show that the covariance matrices of SS-MAMP are Lbanded and convergent. Given an arbitrary MAMP, we can construct the SS-MAMP by damping, which not only ensures the convergence, but also preserves the orthogonality, i.e., its dynamics can be correctly described by state evolution.

show abstract

“…Significantly, it was shown that any L-banded covariance matrix converges in [11], [13], [14], i.e., its diagonal en-tries converge (See Lemma 2). As a result, the convergence difficulties of general iterative algorithms were resolved in principle.…”

Section: Introductionmentioning

confidence: 99%

Algebra of L-Banded Matrices

2023

View full text Add to dashboard Cite

Convergence is a crucial issue in iterative algorithms. Damping is commonly employed to ensure the convergence of iterative algorithms. The conventional ways of damping are scalar-wise, and either heuristic or empirical. Recently, an analytically optimized vector damping was proposed for memory message-passing (iterative) algorithms. As a result, it yields a special class of covariance matrices called L-banded matrices. In this paper, we show these matrices have broad algebraic properties arising from their L-banded structure. In particular, compact analytic expressions for the LDL decomposition, the Cholesky decomposition, the determinant after a column substitution, minors, and cofactors are derived. Furthermore, necessary and sufficient conditions for an L-banded matrix to be definite, a recurrence to obtain the characteristic polynomial, and some other properties are given. In addition, we give new derivations of the determinant and the inverse.

show abstract

Sufficient Statistic Memory AMP

Cited by 3 publications

References 40 publications

Capacity Optimality of OAMP in Coded Large Unitarily Invariant Systems

Capacity Optimality of OAMP in Coded Large Unitarily Invariant Systems

Sufficient Statistic Memory Approximate Message Passing

Algebra of L-Banded Matrices

Contact Info

Product

Resources

About