Statistical Mechanical Approach to Error Exponents of Lossy Data Compression

Abstract

We present herein a scheme by which to accurately evaluate the error exponents of a lossy data compression problem, which characterize average probabilities over a code ensemble of compression failure and success above or below a critical compression rate, respectively, utilizing the replica method (RM). Although the existing method used in information theory (IT) is, in practice, limited to ensembles of randomly constructed codes, the proposed RMbased approach can be applied to a wider class of ensembles. This approach reproduces the optimal expressions of the error exponents achieved by the random code ensembles, which are known in IT. In addition, the proposed framework is used to show that codes composed of non-monotonic perceptrons of a specific type can provide the optimal exponents in most cases, which is supported by numerical experiments.

Statistical mechanical approach to lossy data compression: Theory and practice

“…When the above general framework was applied to the random code ensemble, which is not a practical coding scheme, but can exhibit optimal performance, the theoretical limitationsthe RDF and optimal error exponents derived in IT [4,5] -were reproduced correctly [6,7].…”

The theoretical capacity of the Parity Source Coder

“…A few lines of computation show that the distortion in (19) actually saturates the Shannon bound. Let's call z the value of y where Φ ∞ (y) is maximal.…”

Statistical mechanics of lossy compression using multilayer perceptrons

“…For instance, Low Density Generator Matrix (LDGM) code [7,8] and using a nonmonotonic perceptron [9,10,11] were proposed. In these compression codes, a decoder is first defined to retrieve a reproduced message from a codeword.…”