No abstract

No abstract

No abstract

PACS. 89.90+n -Other areas of general interest to physicists. PACS. 89.70+c -Information science. PACS. 05.50+q -Lattice theory and statistics; Ising problems.Abstract. -We investigate the performance of error-correcting codes, where the code word comprises products of K bits selected from the original message and decoding is carried out utilizing a connectivity tensor with C connections per index. Shannon's bound for the channel capacity is recovered for large K and zero temperature when the code rate K/C is finite. Close to optimal error-correcting capability is obtained for finite K and C. We examine the finite-temperature case to assess the use of simulated annealing for decoding and extend the analysis to accommodate other types of noisy channels.Error-correcting codes are of significant practical importance as they provide mechanisms for retrieving the original message after possible corruption due to noise during transmission. They are being used extensively in most means of information transmission from satellite communication to the storage of information on hardware devices. The coding efficiency, measured in the percentage of informative transmitted bits, plays a crucial role in determining the speed of communication channels and the effective storage space on hard-disks. Rigorous bounds [1] have been derived for the maximal channel capacity for which codes, capable of achieving arbitrarily small error probability, can be found. However, existing practical errorcorrecting codes do not saturate this bound and the quest for more efficient error-correcting codes has been going on ever since.A new family of error-correcting codes, based on insights gained from the statistical mechanical analysis of Ising spin models, has recently been suggested by Sourlas [2], investigating the use of statistical mechanics for constructing and investigating novel coding methods [3,12]. However, the codes suggested and analyzed so far are of no practical significance as they imply an infinite ratio between the length of the transmitted word and the original message. Consequently, they had little impact on the design and the understanding of practical codes.

PACS. 89.70+c -Information science. PACS. 89.90+n -Other areas of general interest to physicists. PACS. 02.50−r -Probability theory, stochastic processes, and statistics.Abstract. -We employ two different methods, based on belief propagation and TAP, for decoding corrupted messages encoded by employing Sourlas's method, where the code word comprises products of K bits selected randomly from the original message. We show that the equations obtained by the two approaches are similar and provide the same solution as the one obtained by the replica approach in some cases (K = 2). However, we also show that for K ≥ 3 and unbiased messages the iterative solution is sensitive to the initial conditions and is likely to provide erroneous solutions; and that it is generally beneficial to use Nishimori's temperature, especially in the case of biased messages.Belief networks [1], also termed Bayesian networks, and influence diagrams are diagrammatic representations of joint probability distributions over a set of variables. The set of variables is usually represented by the vertices of a graph, while arcs between vertices represent probabilistic dependences between variables. Belief propagation provides a convenient mathematical tool for calculating iteratively joint probability distributions of variables, and have been used in a variety of cases to assess conditional probabilities and interdependences between variables in complex systems. One of the most recent uses of belief propagation is in the field of error-correcting codes, especially for decoding corrupted messages [2] (for a review of graphical models and their use in the context of error-correcting codes see [3]).Error-correcting codes provide a mechanism for retrieving the original message after corruption due to noise during transmission. A new family of error-correcting codes, based on insights gained from statistical mechanics, has recently been suggested by Sourlas [4]. These codes can be mapped onto the many-body Ising spin problem and can thus be analysed using methods adopted from statistical physics [5][6][7][8][9].In this letter we will examine the similarities and differences between the belief propagation (BP) and TAP approaches, used as decoders in the context of error-correcting codes. We will then employ these approaches to examine a few specific cases and compare the results to the solutions obtained using the replica method [8]. This will enable us to draw some conclusions on the efficacy of the TAP/BP approach in the context of error-correcting codes.

No abstract

The performance of Gallager's error-correcting code is investigated via methods of statistical physics. In this approach, the transmitted codeword comprises products of the original message bits selected by two randomly-constructed sparse matrices; the number of non-zero row/column elements in these matrices constitutes a family of codes. We show that Shannon's channel capacity is saturated for many of the codes while slightly lower performance is obtained for others which may be of higher practical relevance. Decoding aspects are considered by employing the TAP approach which is identical to the commonly used belief-propagation-based decoding.The ever increasing information transmission in the modern world is based on communicating messages reliably through noisy transmission channels; these can be telephone lines, deep space, magnetic storing media etc. Errorcorrecting codes play an important role in correcting errors incurred during transmission; this is carried out by encoding the message prior to transmission, and decoding the corrupted received codeword for retrieving the original message. In his ground breaking papers, Shannon [1] analyzed the capacity of communication channels, setting an upper bound to the achievable noise-correction capability of codes, given their code (or symbol) rate. The latter represents the ratio between the number of bits in the original message and the transmitted codeword.Shannon's bound is non-constructive and does not provide explicit rules for devising optimal codes. The quest for more efficient codes, in the hope of saturating the bound set by Shannon, has been going on ever since, providing many useful but sub-optimal codes.One family of codes, presented originally by Gallager In this Letter we analyze the typical performance of Gallager-type codes for several parameter choices via methods of statistical mechanics. We then validate the analytical solution by comparing the results to those obtained by the TAP approach to diluted systems and via numerical methods.In a general scenario, a message represented by an N dimensional Boolean/binary vector ξ is encoded to the M dimensional vector J 0 which is then transmitted through a noisy channel with some flipping probability p per bit (other noise types may also be considered but will not be examined here). The received message J is then decoded to retrieve the original message.One can identify several slightly different versions of Gallager-type codes. The one used in this Letter, termed the MN code [3] is based on choosing two randomly-selected sparse matrices A and B of dimensionality M×N and M×M respectively; these are characterized by K and L non-zero unit elements per row and C and L per column respectively. The finite, usually small, numbers K, C and L define a particular code; both matrices are known to both sender and receiver. Encoding is carried out by constructing the modulo 2 inverse of B and the matrix B −1 A (modulo 2); the vector J 0 = B −1 A ξ (modulo 2, ξ in a Boolean representation) constitutes the codeword....

PACS. 89.90+n -Other areas of general interest to physicists. PACS. 89.70+c -Information science. PACS. 05.50+q -Lattice theory and statistics; Ising problems.Abstract. -A variation of low density parity check (LDPC) error correcting codes defined over Galois fields (GF (q)) is investigated using statistical physics. A code of this type is characterised by a sparse random parity check matrix composed of C nonzero elements per column. We examine the dependence of the code performance on the value of q, for finite and infinite C values, both in terms of the thermodynamical transition point and the practical decoding phase characterised by the existence of a unique (ferromagnetic) solution. We find different q-dependencies in the cases of C = 2 and C ≥ 3; the analytical solutions are in agreement with simulation results, providing a quantitative measure to the improvement in performance obtained using non-binary alphabets.Error correction mechanisms are essential for ensuring reliable data transmission through noisy media. They play an important role in a wide range of applications from magnetic hard disks to deep space exploration, and are expected to become even more important due to the rapid development in mobile phones and satellite-based communication.The error-correcting ability comes at the expense of information redundancy. Shannon showed in his seminal work [10] that error-free communication is theoretically possible if the code rate, representing the fraction of informative bits in the transmitted codeword, is below the channel capacity. In the case of unbiased messages transmitted through a Binary Symmetric Channel (BSC), which we focus on here and which is characterized by a bit flip rate p, the code rate R = N/M which allows for an error-free transmission satisfies(