The performance of Gallager's error-correcting code is investigated via methods of statistical physics. In this approach, the transmitted codeword comprises products of the original message bits selected by two randomly-constructed sparse matrices; the number of non-zero row/column elements in these matrices constitutes a family of codes. We show that Shannon's channel capacity is saturated for many of the codes while slightly lower performance is obtained for others which may be of higher practical relevance. Decoding aspects are considered by employing the TAP approach which is identical to the commonly used belief-propagation-based decoding.The ever increasing information transmission in the modern world is based on communicating messages reliably through noisy transmission channels; these can be telephone lines, deep space, magnetic storing media etc. Errorcorrecting codes play an important role in correcting errors incurred during transmission; this is carried out by encoding the message prior to transmission, and decoding the corrupted received codeword for retrieving the original message. In his ground breaking papers, Shannon [1] analyzed the capacity of communication channels, setting an upper bound to the achievable noise-correction capability of codes, given their code (or symbol) rate. The latter represents the ratio between the number of bits in the original message and the transmitted codeword.Shannon's bound is non-constructive and does not provide explicit rules for devising optimal codes. The quest for more efficient codes, in the hope of saturating the bound set by Shannon, has been going on ever since, providing many useful but sub-optimal codes.One family of codes, presented originally by Gallager In this Letter we analyze the typical performance of Gallager-type codes for several parameter choices via methods of statistical mechanics. We then validate the analytical solution by comparing the results to those obtained by the TAP approach to diluted systems and via numerical methods.In a general scenario, a message represented by an N dimensional Boolean/binary vector ξ is encoded to the M dimensional vector J 0 which is then transmitted through a noisy channel with some flipping probability p per bit (other noise types may also be considered but will not be examined here). The received message J is then decoded to retrieve the original message.One can identify several slightly different versions of Gallager-type codes. The one used in this Letter, termed the MN code [3] is based on choosing two randomly-selected sparse matrices A and B of dimensionality M×N and M×M respectively; these are characterized by K and L non-zero unit elements per row and C and L per column respectively. The finite, usually small, numbers K, C and L define a particular code; both matrices are known to both sender and receiver. Encoding is carried out by constructing the modulo 2 inverse of B and the matrix B −1 A (modulo 2); the vector J 0 = B −1 A ξ (modulo 2, ξ in a Boolean representation) constitutes the codeword....
We study an ill-posed linear inverse problem, where a binary sequence will be reproduced using a sparce matrix. According to the previous study, this model can theoretically provide an optimal compression scheme for an arbitrary distortion level, though the encoding procedure remains an NP-complete problem. In this paper, we focus on the consistency condition for a dynamics model of Markov-type to derive an iterative algorithm, following the steps of Thouless-Anderson-Palmer's.Numerical results show that the algorithm can empirically saturate the theoretical limit for the sparse construction of our codes, which also is very close to the rate-distortion function.
A variation of Gallager error-correcting codes is investigated using statistical mechanics. In codes of this type, a given message is encoded into a codeword that comprises Boolean sums of message bits selected by two randomly constructed sparse matrices. The similarity of these codes to Ising spin systems with random interaction makes it possible to assess their typical performance by analytical methods developed in the study of disordered systems. The typical case solutions obtained via the replica method are consistent with those obtained in simulations using belief propagation decoding. We discuss the practical implications of the results obtained and suggest a computationally efficient construction for one of the more practical configurations.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.