Hash property and coding theorems for sparse matrices and maximum-likelihood coding

Abstract

The aim of this paper is to prove the achievability of several coding problems by using sparse matrices (the maximum column weight grows logarithmically in the block length) and maximal-likelihood (ML) coding. These problems are the Slepian-Wolf problem, the Gel'fand-Pinsker problem, the Wyner-Ziv problem, and the Onehelps-one problem (source coding with partial side information at the decoder). To this end, the notion of a hash property for an ensemble of functions is introduced and it is proved that an ensemble of q-ary sparse matrices satisfies the hash property. Based on this property, it is proved that the rate of codes using sparse matrices and maximal-likelihood (ML) coding can achieve the optimal rate. Index TermsShannon theory, hash functions, linear codes, sparse matrix, maximum-likelihood eoncoding/decoding, the Slepian-Wolf problem, the Gel'fand-Pinsker problem, the Wyner-Ziv problem, the One-helps-one problem I. INTRODUCTIONThe aim of this paper is to prove the achievability of several coding problems by using sparse matrices (the maximum column weight grows logarithmically in the block length) and maximal-likelihood (ML) coding 1 , namely the Slepian-Wolf problem [39] (Fig. 1), the Gel'fand-Pinsker problem [13] (Fig. 2), the Wyner-Ziv problem [47] (Fig. 3), and the One-helps-one problem (source coding with partial side information at the decoder) [44][46] (Fig. 4). To prove these theorems, we first introduce the notion of a hash property for an ensemble of functions, where functions are not assumed to be linear. This notion is a sufficient condition for the achievability of coding theorems. Next, we prove that an ensemble of q-ary sparse matrices, which is an extension of [21], satisfies the hash property. Finally, based on the hash property, we prove that the rate of J. Muramatsu is with NTT

Achievability Proof via Output Statistics of Random Binning

“…In fact, the OSRB framework implicitly employs hybrid coding by its construction. This is discussed in details in Remark 8. It was brought to our attention by Muramatsu that a structure similar to OSRB based on random binning (more generally, hash functions) has been used in his works [17,18]. While the use of random binning in the works of Muramatsu et al is similar to ours (in particular its use of binning to get Marton coding), obtaining superposition coding part of Marton's inner bound via binning has been left as an open problem (this can be done by binning nested sets of variables in our framework) [19].…”

Hash Property and Fixed-Rate Universal Coding Theorems

“…In this section, we reveiw the notion of the (α, β)-hash property introduced in [12]. This is a sufficient condition for coding theorems, where the linearity of functions is not assumed.…”

“…The notion of hash property is introduced in [12]. It is a sufficient condition for the achievability of coding theorems including lossless and lossy source coding, channel coding, the Slepian-Wolf problem, the Wyner-Ziv problem, the Gel'fand-Pinsker problem, and the problem of source coding with partial side information at the decoder.…”

“…However, it is assumed in [12] that source and channel distributions are used when designing a code. The aim of this paper is to prove fixed-rate universal coding theorems based on the hash property, where a specific probability distribution is not assumed for the design of a code and the error probability of a code vanishes for all sources specified by the encoding rate.…”

“…2). In the construction of codes, the maximum-likelihood coding used in [12] is replaced by a minimum-divergence encoder and a minimum-entropy decoder. A practical algorithm has been obtained for the minimum-entropy decoder by using linear programming [2].…”