On the joint source-channel coding error exponent for discrete memoryless systems

Abstract

Abstract-We investigate the computation of Csiszár's bounds for the joint source-channel coding (JSCC) error exponent of a communication system consisting of a discrete memoryless source and a discrete memoryless channel. We provide equivalent expressions for these bounds and derive explicit formulas for the rates where the bounds are attained. These equivalent representations can be readily computed for arbitrary source-channel pairs via Arimoto's algorithm. When the channel's distribution satisfies a symmetry property, the bounds admit closed-form parametric expressions. We then use our results to provide a systematic comparison between the JSCC error exponent and the tandem coding error exponent , which applies if the source and channel are separately coded. It is shown that Index Terms-Discrete memoryless sources and channels, error exponent, Fenchel's duality, Hamming distortion measure, joint source-channel coding, random-coding exponent, reliability function, sphere-packing exponent, symmetric channels, tandem source and channel coding.

On Decoder Ties for the Binary Symmetric Channel with Arbitrarily Distributed Input

“…Noting that in (2) is the probability that a decoding error occurs without inducing decoder ties (which occur when two or more codewords in are identified by the decoder as the estimated transmitted codeword; i.e., when more than one codeword in maximize for a given received word ), the above result in (1) directly implies that decoder ties do not affect the error exponent of . The error exponent or reliability function of a block coding communication system represents the largest rate of exponential decay of the system’s probability of decoding error as the coding blocklength grows to infinity (e.g., see [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 ]).…”

“…In this paper, we consider a non-uniform prior distribution over and prove that decoder ties, under optimal MAP decoding, still have linear and hence sub-exponential impact on the error probability , thus extending Theorem 1 established for the case of a uniform prior distribution over . Since our problem falls within the general framework of joint source-channel coding for point-to-point communication systems, we refer the reader to [ 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ] (Section 4.6) and the references therein for theoretical studies on this subject as well as practical designs that outperform separate source and channel coding under complexity or delay constraints.…”

On Decoder Ties for the Binary Symmetric Channel with Arbitrarily Distributed Input

“…Noting that b n in (2) is the probability that a decoding error occurs without inducing decoder ties (which occur when two or more codewords in C n are identified by the decoder as the estimated transmitted codeword; i.e., when more than one codeword in C n maximize P X n |Y n (•|y n ) for a given received word y n ), the above result in (1) directly implies that decoder ties do not affect the error exponent of a n . The error exponent or reliability function of a block coding communication system represents the largest rate of exponential decay of the system's probability of decoding error as the coding blocklength grows to infinity (e.g., see [2]- [14]).…”

“…In this paper, we consider a non-uniform prior distribution over C n and prove that decoder ties, under optimal MAP decoding, still have linear and hence sub-exponential impact on the error probability a n , thus extending Theorem 1 established for the case of a uniform prior distribution over C n . Since our problem falls within the general framework of joint source-channel coding for point-to-point communication systems, we refer the reader to [14]- [20], [21,Section 4.6] and the references therein for theoretical studies on this subject as well as practical designs that outperform separate source and channel coding under complexity or delay constraints.…”