Optimal quantization of B-DMCs maximizing α-mutual information with monge property

Sakai, Yuta; Iwata, Ken-ichi

doi:10.1109/isit.2017.8007013

Cited by 6 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, δ N defined by (3) are sequentially located in a line segment, the optimal SDQ is an optimal DQ and the two efficient techniques are applicable. This result generalizes the results of [16]- [18]. Next, we showed that the cost function of an α-MI-maximizing quantizer can be defined as a specific case of the above general cost function.…”

Section: Discussionsupporting

confidence: 85%

“…For binary-input DMC, dynamic programming (DP) [15,Section 15.3] was applied by Kurkoski and Yagi [16] to design quantizers that maximize the mutual information (MI) between X and Z, i.e., I(X; Z). The complexity (refer to the computational complexity throughout this paper unless the storage complexity is specified) of this DP method was reduced [17], [18] by applying the SMAWK algorithm [19]. However, for the general q-ary input DMC with q > 2, design of the optimal quantizers that maximize I(X; Z) is an NP-hard problem [14], [20].…”

Section: Dmc Quantizermentioning

confidence: 99%

“…Next, we investigate the design of α-mutual information (α-MI) [25]- [27] maximizing quantizer in details, as it attracts a lot of interests recently [2]- [4], [16]- [18], [21]- [23], [28]. We illustrate that the cost function of an α-MI-maximizing quantizer can be defined as a specific case of the general cost function given by (2).…”

Section: Dmc Quantizermentioning

confidence: 99%

“…Moreover, we prove that the two low-complexity techniques are always applicable under the aforementioned sufficient condition that leads the optimal SDQ to be an optimal DQ. Next, we investigate the design of α-mutual information (α-MI) [25]- [27] maximizing quantizer in details, as it attracts a lot of interests recently [2]- [4], [16]- [18], [21]- [23], [28]. We illustrate that the cost function of an α-MI-maximizing quantizer can be defined as a specific case of the general cost function given by (2).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Dynamic Programming for Quantization of q-ary Input Discrete Memoryless Channels

Cai

Song

et al. 2019

2019 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

In this paper, under a general cost function, we present a dynamic programming (DP) method to obtain an optimal sequential deterministic quantizer (SDQ) for q-ary input discrete memoryless channel (DMC). The DP method has complexity O(q(N − M ) 2 M ), where N and M are the alphabet sizes of the DMC output and quantizer output, respectively. Then, starting from the quadrangle inequality (QI), two techniques are applied to reduce the DP method's complexity. One technique makes use of the SMAWK algorithm and achieves complexity O(q(N − M )M ). The other technique is much easier to be implemented and achieves complexity O(q(N 2 − M 2 )). We further derive a sufficient condition under which the optimal SDQ is optimal among all quantizers and the two techniques are also applicable. This generalizes the results in the literature for binary-input DMC. Next, we show that the cost function of α-mutual information (α-MI)-maximizing quantizer belongs to the general cost function we adopt earlier. We further prove that under a weaker condition than the sufficient condition we derived, the aforementioned two techniques are applicable to the design of α-MI-maximizing quantizer. Finally, we propose a new algorithm called iterative DP (IDP). Theoretical analysis and simulation results demonstrate that IDP can improve the quantizer design over the state-of-the-art methods in the literature.Index Terms-α-mutual information (α-MI), discrete memoryless channel (DMC), dynamic programming (DP), quadrangle inequality (QI), sequential deterministic quantizer (SDQ). P Z|Y (z|y) = 1 z = z , 0 z = z , or equivalently, we say y's quantization result Q(y) is a deterministic element in Z. In this paper, a general cost function given by (2) is considered, which is widely adopted in the literature [12]-[14]. Under such a condition, we show that there always exists at least one DQ that is optimal among all quantizers. Due to this reason as well as that DQ is more practical than non-deterministic quantizer, we focus only on DQs in this paper. For any DQ Q : Y → Z, denote Q −1 (z) ⊂ Y as the preimage of z ∈ Z.For binary-input DMC, dynamic programming (DP) [15, Section 15.3] was applied by Kurkoski and Yagi [16] to design quantizers that maximize the mutual information (MI) between X and Z, i.e., I(X; Z). The complexity (refer to the computational complexity throughout this paper unless the storage complexity is specified) of this DP method was reduced [17], [18] by applying the SMAWK algorithm [19]. However, for the general q-ary input DMC with q > 2, design of the optimal quantizers that maximize I(X; Z) is an NP-hard problem [14], [20]. Up till now, only the necessary condition [12], rather than the sufficient condition, has been established for the optimal quantizer; meanwhile, there only exist some suboptimal design methods in practice [5], [14], [21]-[23]. Moreover, to the best of our knowledge, so far no work has arXiv:1901.01659v3 [cs.IT] 28 Oct 2019

show abstract

Section: Discussionsupporting

confidence: 85%

Section: Dmc Quantizermentioning

confidence: 99%

Section: Dmc Quantizermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dynamic Programming for Quantization of q-ary Input Discrete Memoryless Channels

Cai

Song

et al. 2019

2019 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

show abstract

“…A partial list of papers relating to upgrading and degrading approximations is [17]- [30]. Specifically, [26] contains a generalization of the upgrading approximation presented in [5] to cases in which the input has a non-uniform binary distribution.…”

Section: Introductionmentioning

confidence: 99%

An Upgrading Algorithm with Optimal Power Law

Ordentlich¹,

Tal²

2020

Preprint

View full text Add to dashboard Cite

Consider a channel W along with a given input distribution PX . In certain settings, such as in the construction of polar codes, the output alphabet of W is 'too large', and hence we replace W by a channel Q having a smaller output alphabet. We say that Q is upgraded with respect to W if W is obtained from Q by processing its output. In this case, the mutual information I(PX , W ) between the input and output of W is upper-bounded by the mutual information I(PX , Q) between the input and output of Q. In this paper, we present an algorithm that produces an upgraded channel Q from W , as a function of PX and the required output alphabet size of Q, denoted L. We show that the difference in mutual informations is 'small'. Namely, it is O(L −2/(|X |−1) ), where |X | is the size of the input alphabet. This power law of L is optimal.

show abstract