Low Density Codes Achieve theRate-Distortion Bound

Martinian, E.; Wainwright, Martin J.

doi:10.1109/dcc.2006.44

Cited by 37 publications

(44 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We note that our reason for choosing the check-regular LDGM ensemble specified in step 1) is not that it necessarily defines a particularly good code, but rather that it is convenient for theoretical purposes. Interestingly, our analysis shows that the bounded degree check-regular LDGM ensemble, even though it is suboptimal for both source and channel coding in isolation [33], [34], defines optimal source and channel codes when combined with a bottom LDPC code.…”

Section: A Compound Constructionmentioning

confidence: 93%

“…In the limit of zero distortion, this analysis has been made rigorous in a sequence of papers [11], [13], [16], [37]. Moreover, our own recent work [32], [33] provides rigorous upper bounds on the effective rate-distortion function of various classes of LDGM codes, whereas Dimakis et al [15] provide rigorous lower bounds on the LDGM rate-distortion function. In terms of practical algorithms for lossy binary compression, researchers have explored variants of the sum-product algorithm [39] or survey propagation algorithms [9], [51] for quantizing binary sources.…”

Section: A Previous and Ongoing Workmentioning

confidence: 99%

See 1 more Smart Citation

Low-Density Graph Codes That Are Optimal for Binning and Coding With Side Information

Wainwright

Martinian²

2009

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Abstract-In this paper, we describe and analyze the source and channel coding properties of a class of sparse graphical codes based on compounding a low-density generator matrix (LDGM) code with a low-density parity-check (LDPC) code. Our first pair of theorems establishes that there exist codes from this ensemble, with all degrees remaining bounded independently of block length, that are simultaneously optimal for both channel coding and source coding with binary data when encoding and decoding are performed optimally. More precisely, in the context of lossy compression, we prove that finite-degree constructions can achieve any pair (R; D) on the rate-distortion curve of the binary symmetric source. In the context of channel coding, we prove that the same finite-degree codes can achieve any pair (C; p) on the capacity-noise curve of the binary symmetric channel (BSC). Next, we show that our compound construction has a nested structure that can be exploited to achieve the Wyner-Ziv bound for source coding with side information (SCSI), as well as the Gelfand-Pinsker bound for channel coding with side information (CCSI). Although the results described here are based on optimal encoding and decoding, the proposed graphical codes have sparse structure and high girth that renders them well suited to message passing and other efficient decoding procedures.

show abstract

Section: A Compound Constructionmentioning

confidence: 93%

Section: A Previous and Ongoing Workmentioning

confidence: 99%

Low-Density Graph Codes That Are Optimal for Binning and Coding With Side Information

Wainwright

Martinian²

2009

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…In the limit of zero-distortion, this analysis has been made rigorous in a sequence of papers [5], [15], [4], [6]. Moreover, our own recent work [11], [10] provides rigorous upper bounds on the effective rate-distortion function of various classes of LDGM codes. In terms of practical algorithms for lossy binary compression, researchers have explored variants of the sum-product algorithm [16] or survey propagation algorithms [2], [18] for quantizing binary sources.…”

Section: Past Workmentioning

confidence: 99%

“…Our contributions: Previous analysis of LDGM ratedistortion [11], [12] was based on the first and second-moment methods from probabilistic combinatorics [1]. Whereas the second moment provides a non-trivial upper bound on the effective rate-distortion function, the first moment method yields a well-known statement-namely, the Shannon bound, which is far from sharp for these sparse graph codes.…”

Section: Past Workmentioning

confidence: 99%

Lower bounds on the rate-distortion function of LDGM codes

Dimakis

Wainwright

Ramchandran

2007

2007 IEEE Information Theory Workshop

View full text Add to dashboard Cite

Abstract-We analyze the performance of low-density generator matrix (LDGM) codes for lossy source coding. We first develop a generic technique for deriving lower bounds on the effective rate-distortion functions of binary linear codes. This result provides a source coding analog of a classical result due to Gallager for channel coding over the binary symmetric channel. We illustrate this method for the ensemble of check-regular lowdensity generator matrix (LDGM) codes by deriving an explicit lower bound on its rate-distortion performance as a function of the check degree.

show abstract

“…1 to obtain a lossy compressor. Sparse-graph code based compressors are constructed in [12] and [13] which are optimal for compressing the binary symmetric source with Hamming distortion. In [14] and [15] asymptotically optimal sparse graph codes are constructed for the problem of compressing the -ary source with Hamming distortion, and the discrete memoryless source with separable distortion respectively.…”

Section: Sparse-graph Based Schemesmentioning

confidence: 99%

Operational Duality Between Lossy Compression and Channel Coding

Gupta

Verdú

2011

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Abstract-We explore the duality between lossy compression and channel coding in the operational sense: whether a capacity-achieving encoder-decoder sequence achieves the rate-distortion function of the dual problem when the channel decoder [encoder] is the source compressor [decompressor, resp.], and vice versa. We show that, if used as a lossy compressor, the maximum-likelihood channel decoder of a randomly chosen capacity-achieving codebook achieves the rate-distortion function almost surely. However, operational duality does not hold for every capacity achieving encoder-decoder sequence, or rate-distortion achieving compressor-decompressor sequence. We show that there exist optimal channel coding [lossy compression] schemes, which fail when used for the dual lossy compression [channel coding resp.] problem.

show abstract

Low Density Codes Achieve theRate-Distortion Bound

Cited by 37 publications

References 17 publications

Low-Density Graph Codes That Are Optimal for Binning and Coding With Side Information

Low-Density Graph Codes That Are Optimal for Binning and Coding With Side Information

Lower bounds on the rate-distortion function of LDGM codes

Operational Duality Between Lossy Compression and Channel Coding

Contact Info

Product

Resources

About