Low-Density Graph Codes That Are Optimal for Binning and Coding With Side Information

Abstract

Abstract-In this paper, we describe and analyze the source and channel coding properties of a class of sparse graphical codes based on compounding a low-density generator matrix (LDGM) code with a low-density parity-check (LDPC) code. Our first pair of theorems establishes that there exist codes from this ensemble, with all degrees remaining bounded independently of block length, that are simultaneously optimal for both channel coding and source coding with binary data when encoding and decoding are performed optimally. More precisely, in the context of lossy compression, we prove that finite-degree constructions can achieve any pair (R; D) on the rate-distortion curve of the binary symmetric source. In the context of channel coding, we prove that the same finite-degree codes can achieve any pair (C; p) on the capacity-noise curve of the binary symmetric channel (BSC). Next, we show that our compound construction has a nested structure that can be exploited to achieve the Wyner-Ziv bound for source coding with side information (SCSI), as well as the Gelfand-Pinsker bound for channel coding with side information (CCSI). Although the results described here are based on optimal encoding and decoding, the proposed graphical codes have sparse structure and high girth that renders them well suited to message passing and other efficient decoding procedures.

Spatially-coupled MacKay-Neal codes and Hsu-Anastasopoulos codes

“…We name the codes HsuAnastasopoulos (HA) codes after the inventors. Furthermore, Wainwright and Martinian showed HA codes achieve the rate-distortion bound for symmetric Bernoulli sources [23].…”

Polar codes are optimal for lossy source coding

“…In this respect, Matsunaga and Yamamoto [5] showed that if the degrees of a low-density parity-check (LDPC) ensemble are chosen as large as Θ(log(N )), where N is the blocklength, then this ensemble saturates the ratedistortion bound if optimal encoding is employed. Even more promising, Martininian and Wainwright [6] proved that properly chosen MN codes with bounded degrees are sufficient to achieve the rate-distortion bound under optimal encoding.…”

Lossy Source Compression Using Low-Density Generator Matrix Codes: Analysis and Algorithms

“…Since portions of this research were first published in conference form [46], [29], other work has followed up and extended some of our results. Martinian and Wainwright [30], [31], [47] have studied the use of compound constructions, in which an LDGM code is concatenated with an LDPC code, for both lossy compression and binning. Gupta and Verdu [21] have used similar analysis techniques (moment methods) to study the performance of a novel non-linear scheme for lossy compression.…”