This paper develops systematic approaches to obtain f -divergence inequalities, dealing with pairs of probability measures defined on arbitrary alphabets. Functional domination is one such approach, where special emphasis is placed on finding the best possible constant upper bounding a ratio of f -divergences. Another approach used for the derivation of bounds among f -divergences relies on moment inequalities and the logarithmic-convexity property, which results in tight bounds on the relative entropy and Bhattacharyya distance in terms of χ 2 divergences. A rich variety of bounds are shown to hold under boundedness assumptions on the relative information. Special attention is devoted to the total variation distance and its relation to the relative information and relative entropy, including "reverse Pinsker inequalities," as well as on the E γ divergence, which generalizes the total variation distance. Pinsker's inequality is extended for this type of f -divergence, a result which leads to an inequality linking the relative entropy and relative information spectrum. Integral expressions of the Rényi divergence in terms of the relative information spectrum are derived, leading to bounds on the Rényi divergence in terms of either the variational distance or relative entropy.

The complete characterization of the capacity region of a two-user Gaussian interference channel is still an open problem unless the interference is strong. In this work, we derive an achievable rate region for this channel. It includes the rate region which is achieved by time/ frequency division multiplexing (TDM/ FDM), and it also includes the rate region which is obtained by time-sharing between the two rate pairs where one of the transmitters sends its data reliably at the maximal possible rate (i.e., the maximal rate it can achieve in the absence of interference), and the other transmitter decreases its data rate to the point where both receivers can reliably decode its message. The suggested rate region is easily calculable, though it is a particular case of the celebrated achievable rate region of Han and Kobayashi whose calculation is in general prohibitively complex. In the high power regime, a lower bound on the sum-rate capacity (i.e., maximal achievable total rate) is derived, and we show its superiority over the maximal total rate which is achieved by the TDM/ FDM approach for the case of moderate interference. For degraded and one-sided Gaussian interference channels, we rely on some observations of Costa and Sato, and obtain their sum-rate capacities. We conclude our discussion by pointing out two interesting open problems.

No abstract

No abstract

No abstract

Low-density parity-check (LDPC) codes are efficiently encoded and decoded due to the sparseness of their parity-check matrices. Motivated by their remarkable performance and feasible complexity under iterative message-passing decoding, we derive lower bounds on the density of parity-check matrices of binary linear codes whose transmission takes place over a memoryless binary-input output-symmetric (MBIOS) channel. The bounds are expressed in terms of the gap between the rate of these codes for which reliable communications is achievable and the channel capacity; they are valid for every sequence of binary linear block codes. For every MBIOS channel, we construct a sequence of ensembles of regular LDPC codes, so that an upper bound on the asymptotic density of their parity-check matrices scales similarly to the lower bound. The tightness of the lower bound is demonstrated for the binary erasure channel by analyzing a sequence of ensembles of right-regular LDPC codes which was introduced by Shokrollahi, and which is known to achieve the capacity of this channel. Under iterative message-passing decoding, we show that this sequence of ensembles is asymptotically optimal (in a sense to be defined in this paper), strengthening a result of Shokrollahi. Finally, we derive lower bounds on the bit error probability and on the gap to capacity for binary linear block codes which are represented by bipartite graphs, and study their performance limitations over MBIOS channels. The latter bounds provide a quantitative measure for the number of cycles of bipartite graphs which represent good error-correction codes.

The paper introduces ensembles of accumulate-repeat-accumulate (ARA) codes which asymptotically achieve capacity on the binary erasure channel (BEC) with bounded complexity, per information bit, of encoding and decoding. It also introduces symmetry properties which play a central role in the construction of capacity-achieving ensembles for the BEC with bounded complexity. The results here improve on the tradeoff between performance and complexity provided by previous constructions of capacity-achieving ensembles of codes defined on graphs. The superiority of ARA codes with moderate to large block length is exemplified by computer simulations which compare their performance with those of previously reported capacity-achieving ensembles of LDPC and IRA codes. The ARA codes also have the advantage of being systematic.

We present two sequences of ensembles of non-systematic irregular repeat-accumulate codes which asymptotically (as their block length tends to infinity) achieve capacity on the binary erasure channel (BEC) with bounded complexity per information bit. This is in contrast to all previous constructions of capacity-achieving sequences of ensembles whose complexity grows at least like the log of the inverse of the gap (in rate) to capacity. The new bounded complexity result is achieved by puncturing bits, and allowing in this way a sufficient number of state nodes in the Tanner graph representing the codes. We also derive an informationtheoretic lower bound on the decoding complexity of randomly punctured codes on graphs. The bound holds for every memoryless binary-input output-symmetric channel and is refined for the BEC.